In a large mixing bowl, beat the heavy whipping cream until soft peaks form.
In another bowl, beat the softened cream cheese until smooth.
Gradually add the powdered sugar and vanilla extract, mixing until well combined.
Gently fold the whipped cream into the cream cheese mixture until fully incorporated.
In a separate bowl, combine the lemon curd and milk, stirring until smooth.
In a trifle dish or a large glass bowl, layer half of the ladyfinger cookies at the bottom.
Drizzle half of the lemon curd mixture over the cookies.
Spread half of the cream cheese mixture on top, then add half of the blueberries and raspberries.
Repeat the layers with the remaining ingredients, finishing with a layer of cream cheese mixture on top.
Refrigerate the trifle for at least 2 hours to allow the flavors to meld and the ladyfingers to soften.
Before serving, sprinkle the lemon zest over the top for added flavor and garnish with extra berries if desired.