Preheat the oven to 350 and grease a casserole dish. (You can use a 9x13 casserole dish or a deep 9x9 casserole dish.)
Cut Brioche loaf into 1/2 inch cubes. Spread out 2/3 of them in the casserole dish. Set 1/3 aside for a little bit.
Beat cream cheese, sugar, lemon zest, vanilla extract, and heavy whipping cream in a mixer until smooth and fluffy.
Transfer cheesecake mixture into a piping bag (or a zip-lock bag), cut off a the tip to make a small opening and drizzle cheesecake mixture over the bread crumbs. Try to get all the areas.
Spread remaining 1/3 of bread crumbs over the cheesecake drizzle.
In a large mixing bowl, whisk together eggs, milk, heavy whipping cream, sugar, lemon extract, and vanilla extract. Whisk well and pour it all over the bread.
Crumble:
In a medium bowl, combine brown sugar, white sugar, lemon zest and flour. Mix well until all ingredients are incorporated evenly.
Pour in melted butter and carefully fold it in with a large fork. You will want to be careful so you don’t over-mix. Streusel should look like large crumbs and not paste.