... I would like to know if there is any delay apart from hardware ...
... I am curious to know the latency in milliseconds ...
I don't think anyone can give you a definitive answer. We can all only guess.
Shawn was your best hope as he developed much of XNA and he has already answered with, no doubt, as much information as he has.
Neither Windows, nor XNA are designed for the type of input you are describing. Windows apps are typically event driven but the keyboard is buffered and designed to avoid key bounces sending double characters so what you get timing wise is meaningless. If I press the same key twice you may get the event twice but have to handle one before the other so your own code introduces latency. If I repeat press the key too fast one of the presses will be ignored because it looks to the computer like a bounced key.
XNA is better because you have a loop usually of a known speed but it reads the state at a point in time. If the end user is super fast they could press the keyboard multiple times between reading the state.
In the latter case you could try to get your Update() loop to be as often as you can (use the stopwatch class to measure it) and get the state of the keyboard as often as you can unrelated to how often you draw to the screen. That still only captures as many keypresses as possible it does not tell you the real time the key was pressed.
I don't think you want to hear this but I think you need to look again at what you are trying to achieve and do it a different way. Very accurate timing using a normal computer is a fallacy.