Went to the Priory as my Daughter was doing a project for school. Went to the top of the Church, bit of effort for someone as unfit as me, but well worth the effort if you ever go. Some pictures
I've never seen a Vyse Street on any map of that area. The roadway you show in the picture is the entrance to St Mary's Church and the Priory and I don't think it ever led anywhere else. However, it may well have been called Vyse Street.
The area south of this roadway (approximately where you've made the red marking) used to be the quite extensive church graveyard. This was cleared to make way for Cammell Laird's new dry dock, which opened in April 1962.
There used to be a Mersey Street just near here, running down to the riverfront by the old Birkenhead ferry.
Maybe you have seen this site previously, it doeas have a fair amount of history relating. Look down the right hand side for various pages of information about the Priory.(not necessarily in order) This page tells about the graves. Can't see a mention of Vyse Street on any of the pages.
As 5,500 bodies were removed, one wonders where they all ended up (not all intered at Landican, I shouldn't have thought!)
Humankind has not woven the web of life. We are but one thread within it. Whatever we do to the web, we do to ourselves. All things are bound together. All things connect. ~Chief Seattle