I was digging deep to help remove the insane thistle weed root system when I realized beneath the soil on the edge of the driveway there were bricks. (mind you, it has always been jagged and uneven)

Is there a way to even out the asphalt so that if I were to add pavers, the driveway borders would look neat? Do I need to trim down the edges with a saw?