Beat cream cheese, granulated sugar, egg yolks, vanilla extract and lemon zest & lemon juice together in a large bowl. Set aside.
Combine half & half and milk in a medium-large saucepan; bring the mixture to a boil over med heat, stirring occasionally.
Place pan in a large bowl of ice for about 5 minutes, stirring occasionally (this helps cool the mixture down quickly).*
You can allow the mixture to cool on its own to room temperature. It will probably take about 20-30 minutes or so (and don’t forget to whisk occasionally). Basically, you want the custard to come to room temperature before refrigerating.