As best I can tell, all clearances are within spec, however the #2 exhaust clearances are at the bottom of the spec.
If the clearances are (were) in spec, then why did you go any further? To my mind "in spec" is "in spec". I think people get hung up on being in the middle or on the loose end of spec. It seems to me that the manufacturer takes into consideration service interval and wear when they come up with the spec. If you are on the bottom of spec this time and next time you are below spec, this is part of the grand design. I think being slightly out of spec is not dangerous to the motor, but it is at that point that an adjustment is should be made.
If you do have valves out of spec and have to take the cams out, then it makes sense to do some shim swapping to move away from the edges of spec, which might keep you from ever having to remove the cams again.
But removing the cams when the valves are all in spec opens the door to mistakes like you encountered.
By the way, I currently have my cams out, waiting for mail-order shims. This is my first check and I had 3 valves out of spec, and 4 that were at the bottom of spec. I'm only correcting the the 3, as I don't want to spend additional $35 on shims at this time. That may end up being a bad choice; we'll see next time.
I also had one of those dowels go AWOL, but I saw it when it went flying and saw where it landed.
One of the things I learned in removing the cams is that there are 2 different positions for the cams when the engine is at the "2 over T" mark. (Similar to the issue you had when making your initial measurements a couple of days ago). On one of those positions, the cams are under tension from the valve springs, but rotating the engine 360 degrees apparently leaves them with all the lobes sticking up, away from the valves. This latter position makes removal of the cam caps easier and safer as they should come off without the tension of the valve springs. Unfortunately, I didn't recognize this difference and put the engine in the "wrong" position and the cam caps came off with some difficulty.
So, the moral of the story is to position the motor at the 2 over T mark with the cams positioned in the "reassembly position" that the manual calls for. This should make the cam cap removal easy.
The weird thing is, that I will be putting the cams in in a different position than they were when I took them out, but the engine won't know or care. With the cams removed, there is no longer a correct and incorrect 2 over T position, correctness is determined by the re-installation of the cams.
It sounds like we are at about the same stage in our respective valve adjustments. Good luck with yours!