It's an expression. Even though he isn't physically with you, because you are so close to each other and can remember him, it's sort of like being being with you because you can guess at how he would react to certain situations here. It doesn't translate very well though.