Google Marketing Live 2025: AI in Search & Measurement Trends

[00:00] hey Simon jim how are you i'm doing good i'm doing well you know it's uh it's uh the week

[00:08] after a certain event took place that we're going to be talking about there's a lot of excitement in the

[00:13] air we are we are yeah i just flew in from Google Marketing Live and uh my arms are tired

[00:20] i was just going to make the same joke yeah it's hard it's ridiculous i need more dad jokes uh

[00:24] being you'd think being a father I would have come up with more dad jokes over the years but I

[00:28] actually I' I've just cons consolidated down to like three that I just tell all the time now so that's

[00:34] the that's the life of getting little sleep but I got this cool shirt so if you're watching the video

[00:38] stream you can see wearing a data guy shirt uh which they were just given out randomly uh GML but

[00:43] I think that is a little bit of a prelude because I did talk a lot about measurement this year

[00:46] which is which is pretty atypical for these events so that's what I think that's what we're going to talk

[00:51] about today right so yeah so Google Mark Google IO and Google Marketing Live this year they come together normally

[00:56] they have them out um but this year they were together and lots of announcements lots of AI and you

[01:04] were lucky enough to be there right in this and getting you know all of the announcements and obviously some

[01:10] some NDA stuff that you can't reveal to us but uh that's right we'll have to edit anything out that's

[01:17] No I I'm I'm pretty practiced with the with the NDAs these days so I I I I'm I'm usually

[01:21] pretty good about um what I can and cannot speak about uh but yes I was there at GML um

[01:26] last year for anyone who follows me on Twitter you may or LinkedIn um I met Shaq it was a

[01:31] bizarre scenario where Shaq uh I was sitting in the front row and I was just smiling just like enjoying

[01:36] what he was talking about because he's talking about like being this like serial entrepreneur i was like "Ah this

[01:40] is great." And it's Shack right like he's just a he brings positive energy to the room uh and he

[01:44] was just like "You're a good-look dude." And I was like "You're a good looking." and and then it just

[01:49] escalated from there and then he asked me up on stage um and so so it did that didn't happen

[01:54] this year um T Payne was there interestingly um not speaking but Tay did like uh he did a couple

[02:00] songs he did a I have to ask who is who is T Payne you don't know who Tay is

[02:06] i don't know oh man okay well Tay uh you may actually most famously know him for singing I'm on

[02:11] a boat uh with the Lonely Island on SNL you don't know I'm on a boat oh my god oh

[02:17] my god jim what hap what it's like yeah we used other songs that I can't say the the titles

[02:25] of or else we'll have to put the explicit warning on this sure sure so there's I'm on a Yeah

[02:30] yeah well so I'm on a boat um you may also know him recently from doing a cover of Lil

[02:36] John um Get Low uh with the one and only Mark Zuckerberg uh that was the Mark Zuckerberg it was

[02:44] the Zpane that came out last year uh that I don't know if you heard that but it was Mark's

[02:48] song to um Priscilla his wife uh about uh his love for her and that is not a love song

[02:54] it is it is uh it's not a safe for work or a safe pod song um that is immediately

[03:01] going to get us um bleeped or or whatever de I say demonetized we're not even monetized we have nothing

[03:06] to worry about all right so the key takeaway from this is that Jim needs to go catch up on

[03:12] Yes on uh current I this isn't even current events the these are like 10 to 15 years ago uh

[03:19] but but we are the the take away from this is that there were a number of measurement announcements measurement

[03:24] announcements Google marketing live measurement that's right that's right and RIP to or just like I had a lot of

[03:29] empathy for the Google events team having to do IO and Gmail at the same time but it actually worked

[03:32] out incredibly well um I think one of the really interesting things we saw was the um the future of

[03:38] search and I don't even know if search is is is the appropriate term I think that's almost perhaps overly

[03:42] reductive as how we think about the agentic nature of discovery moving forward and I'm using a lot of buzzwords

[03:47] but you'll start to hear these a lot more um and that really does change how we think about the

[03:51] measurement of search and that is a precursor to an episode that we're going to have I think coming out

[03:56] maybe next week or the week after um where we'll have Mike King otherwise known as Poolool Rank who is

[04:01] also a rapper um he's going to be he's going to be on the show um talking about uh SEO

[04:06] measurement because it's important it's something that we haven't covered too much in the past i think in large part

[04:11] because it it it's mm adjacent it's you know it's just oh that's part it's there like I know but

[04:18] I can't necessarily bring into this model i don't know how effective it is and the question is is it

[04:22] effective is the future of surge going to be effective um you know is it can we can we measure

[04:26] it effectively uh but that's all what we're talking about we are talking about the three big announcements that came

[04:31] out at GML um and there was like I said they gave out these shirts uh there was one discussion

[04:37] of um confidential computing which is the the trusted execution environments um proposal if you Google trust execution environments Simon

[04:44] Pton I have a whole big blog about this on the tenity website um that you can go and read

[04:48] about but it's essentially the way that um data portability almost like a clean room layer that is applied to

[04:53] data portability on top of Google so if you want to bring in um custom audiences and so forth without

[04:57] actually exposing ID that's how they think about it in terms of cohort based um portability one mention of that

[05:04] it was one mentioned last year too i have a I have a ticker because I'm like I want to

[05:07] hear more about that thing now I also appreciate that most people in the audience don't want to hear about

[05:10] what they do want to hear about it is incrementality because that is such a buzzword in our industry we

[05:16] did a show about incrementality a while back um with the with folks from measured um which was really well

[05:21] received from from what we heard from it from everyone and this interesting problem of incrementality came up then as

[05:26] to do you trust incrementality testing that is run in the platform and often times the defa default answer is

[05:31] going to be no but I think there's this interesting balance of well how much of a measurement purist are

[05:36] you uh in the sense of everything must be exact science all the time versus how much of a marketingminded

[05:43] sort of we got to be agile and we got to move quickly And we kind of just need some

[05:46] directional insight here and that is the the nature of incrementality testing within platforms uh and I I think one

[05:53] of the the hurdles and we talked about this as well with with the team for measured is the the

[05:57] the cost or you know the loss but also the the the cost if you're doing a heavy up of

[06:02] incrementality testing um and the the announcement and this Jim I when I heard this I could ask Jim about

[06:07] that because I I'm like how how is this going to function um they said and this was and I

[06:12] quote "Based on customer feedback we're making incrementality testing more accessible in several ways by lowering the budget to run

[06:19] a test to only 5K switching to a Beijian model so the tests show more conclusive results more often allowing

[06:26] testing across all campaign types and across accounts and enabling marketers to see those results directly in the Google Ads

[06:31] UI." And I heard that and I said "How are they doing it?" Like one I looked I I said

[06:36] 5k if you're in like uh uh uh I don't know um homeowners insurance or like uh legal like in

[06:44] in the legal field your CPCs could be $200 that is you're talking about 25 clicks here this is this

[06:52] is there's no way the statistical significance there um but the the one big thing they said you know we've

[06:57] introduced more transparency and reporting results at at certainty of lift below 90% as well so advertisers can see those

[07:03] direction results what does this all mean Jim what can you help me understand what they're talking i mean I

[07:09] know I think I know what they're talking about from a from a measurementminded leader what are they is this

[07:16] like is this plausible is this optical what is it yeah i mean you raised the big question it's like

[07:22] okay the minimum to run a conversion lift study is 5,000 but if you're in the messothelomia space for lawyers

[07:29] yes that might get you like what two clicks or whatever the cost per click of that is and so

[07:34] like clearly that's not testable so there's got to be some sort some sort of guard rails around that I'm

[07:39] sure uh the interesting thing to me is that they're switching to a basian methodology which is like you mean

[07:45] you weren't before right surprising like what were you doing before i don't know what their methodology was beforehand but

[07:52] moving to a Beijing methodology is interesting uh the one thing that caught my eye caught my eye was the

[07:57] um introducing a basian methodology which relies on advertisers prior experiments and testing Google has conducted so that tells me

[08:05] that a if you've run previous experiments they're going to somehow be pulling in those signals when they're setting priors

[08:11] and things like that uh and also they're using their vast knowledge of all the experiments they've run and probably

[08:19] segmented by the same type of uh industry same type of market size of customer like probably doing it in

[08:26] a way that's like not just like here's the average whatever for all of our tests it's you know segmenting

[08:31] it properly but you know if they're kind of doing that that that's smart um I can see how that

[08:36] would potentially speed things up where you don't have to let the test run as long but still the whole

[08:41] $5,000 minimum barrier i don't know if that's just uh trying to get more people to be aware of it

[08:52] whereas before maybe people thought "Oh I can never do this testing it's too expensive." Yeah even look at it

[08:58] whereas there could be some tests where $5,000 would be enough right

[09:05] it may be an average um it I I struggle with this cuz I'm to I think I think you're

[09:11] exactly right in terms how they thought about it optically but I'm like yeah but like I would just say

[09:14] the number of key events right like that's what you really want we're lowering the threshold to uh 50 key

[09:19] events within 30 days or something like that we like that's incredibly low still and actually that's a lot more

[09:23] applicable to mind business where we have all this varying varying CPCs and therefore varying cost per acquisition in in

[09:30] in in our space because even if when we're talking CPC I'm talking 25 25 clicks that's not actually even

[09:36] telling me if they're converting clicks and if we say the converting there is 10% and that's being really generous

[09:42] you're looking at 2.5 conversions um so that that's I I I I don't see how that is going to

[09:47] be possible um but this did also click in my brain when I was thinking about are you familiar with

[09:53] you know how how DDA worked in in Google in Google Ads yeah yeah so I know how it worked

[09:58] in Google Analytics you know so same the same shley value model yeah yeah um the when they brought it

[10:06] to market it was there was a very high threshold in terms of the number of conversions that you needed

[10:11] to have on a monthly basis or past 90-day basis which excluded some brands especially like brands that had like

[10:17] extreme seasonality where they're just like we just don't have that volume in the summer and so they were they

[10:22] were excluded out of it and the thought there was they they lower they kept on lowering and lowering and

[10:26] lowering that threshold and there was always the well how is how can you be how do you have confidence

[10:31] if you keep lowering the threshold but then my whole thought was like it's because they have like a a

[10:36] model of models and so they are uh essentially this is not exact science so much as is just it

[10:42] it looks like the model for this all these ones where we did have scale to to correctly or to

[10:48] accurately provide EDA now we see on the if it starts to look like that at the beginning then we

[10:54] can apply DDA here and I'm wondering do you think that's what they cuz I know they're calibrating with prize

[10:57] do you think that's what they're doing here like they're just looking at the model to say this looks like

[11:01] it's going to be significant and then extrapolating i mean that's that is interesting it's hard to say uh you

[11:08] kind of I might go down a rabbit hole around the whole model of models thing because Okay please do

[11:13] this with my tinfoil hat stuff well this is where I kind of started souring or continued souring on multi-touch

[11:21] attribution even with datadriven attribution DDA is that you know as privacy control started proliferating and GDPR and ITP and

[11:30] all that fun stuff you know Google was starting to miss conversions because people didn't provide consent or whatever they

[11:38] they weren't in the data and so Google was now modeling conversions which is then feeding into the modeling of

[11:45] the DDA so you basically a model based on output from a model if you've ever made a copy of

[11:52] a copy of a copy of a copy you know that a lot gets lost during that transmission and I

[11:59] like to think about it with like LLM image generation for like the you're like that's a woman now that's

[12:04] not even my face like how much it changes exactly so like you know you're just compounding the error right

[12:12] from one model you take the output and feed it into another model any error that you had in the

[12:15] first model is going to be exacerbated in the second modelification yeah exactly so like at that point I'm like

[12:23] you know the more the the the more that you know privacy and GDPR and and tech like Apple introduces

[12:32] reduces the actual data that you have the more Google is trying to model it the more air it's going

[12:37] to have which then it's going to be feeding more air into the next model it's like at a certain

[12:42] point you think there there's some threshold some unknown threshold beyond which it's garbage yeah there's a certain point where

[12:49] like okay this could still be kind of directionally right but once you reach a certain level I have a

[12:54] hard time believing that any kind of model data feeding into a model of attribution is worth anything at all

[13:01] which then surprises me and maybe we can move on to this which is the other uh announcement that they

[13:07] made some of the announcements they made around uh Google Analytics around seemingly like doubling down on multi-touch attribution and

[13:15] cross channel performance and yeah stuff like that so maybe you can talk a little bit about that absolutely yeah

[13:20] and and I think uh I don't know the the running joke with Google Analytics has always been if you

[13:25] make a feature request 5 years ago then you'll see it now on so it's one of those like did

[13:29] this come from a long time ago is that why it's now coming and I'm sure there's a lot of

[13:34] um look I we talked about Google Analytics with um that episode with Dana Dtomaso a while back where it's

[13:39] like you're stuck with it now and it wasn't necessarily that part of it was that the data wasn't there

[13:45] but also it was just like how the data is structured and how it is accessible and those and you're

[13:50] like there's a lot of things you can do to improve this ecosystem itself a lot of uh quality of

[13:54] life measures that I think a lot of folks would appreciate but obviously they're trying to push trucks to to

[13:57] BigQuery um to have that that direct connect but I think what is particularly interesting and maybe they're doing this

[14:02] in in perhaps response to the the north beams and the triple whales of the world where they are essentially

[14:07] taking um the modeling of view through conversions from other platforms like um meta for example um and what they

[14:14] what they came out with this is the the big announcement from uh from from Gmail about Google Analytics was

[14:19] marketers will and this is a quote marketers will be able to effectively measure the entire customer journey across Google

[14:26] and other platforms including impressions with enhanced MTA uh this means enabling new metrics like view through conversions and deep

[14:34] insights into marketing ROI and with privacy protections by default right well this is where it may be true because

[14:43] again it is modeled um we don't know like this is not necessarily deterministic uh in in nature but they

[14:50] said the goal is to give businesses a confident picture of their customer journey to help make them uh to

[14:55] help make smarter campaign and budgeting decisions um they're going to be working through partnerships with other platforms um toward

[15:01] a northstar and I think what's particularly interesting about this because I mentioned Meta up top was the three partners

[15:06] that they have and they said there's more to come but the three they're launching with are Tik Tok Snap

[15:11] and Pinterest tik Tok's a big one tik Tok is a big one i was and I was to see

[15:16] that i was like shocked because I read that line about they're working with a plat we welcome partnerships with

[15:22] any platforms i'm like "Oh yeah Google i'm sure all the platforms are going to run to you and say

[15:26] "Here's our data why don't you use it?" Like come on right right and I'm like huh what and and

[15:32] then yeah there's a question well what's in it for them and I think what's in it for them is

[15:34] that there are still a lot of um CFOs and CMOs out there who have this difference to Google Analytics

[15:40] and they say well did Google Analytics say and we saw this when I was complaining about it in our

[15:44] big bets for the year when I was saying why is Meta bringing this to market where they allow you

[15:48] to plug in Google Analytics as your optimization source of truth meaning this is what I'm going to target as

[15:54] clicks in Google Analytics which is by the way why I thought it was also interesting that meta was not

[15:58] included in this because knowing that meta has said this is actually part of your model like don't you want

[16:02] you through meta don't you want those going back into Google uh but there are major competitive constraints and considerations

[16:08] here and it does rely I think very heavily on Google having some kind of ID on those individuals because

[16:13] they have to be able to unify that view through back to the conversion point because man they do have

[16:18] their own I like maybe there's an identity spine that they're sharing through like maybe it is trusted execution environments

[16:23] who knows um but I I looked at this and I thought gosh I wanted that five year no six

[16:30] seven years ago i wanted this when I was an MTA absolutist i don't know how much value it brings

[16:36] now if anything I think it just cloudies and muddies this picture a little bit more where look it's better

[16:41] than just having Google impressions in there which was the default until now right there was always this like it's

[16:46] going to always overcome Google but there is this inherent problem that I I look at and go I don't

[16:51] know what I do with this information now what is the inherent value of having these in here when Google's

[16:56] also really pushing their mm and we'll talk about that in a minute but but I'd love your take on

[17:01] just like is this valuable is it even feasible what what do you think's going on here yeah I mean

[17:08] so let's let's assume for a second that Meta jumps on board and says yeah we'll share our impression data

[17:13] with Google and somehow Google will be able to tie those to an identaph and say this user saw this

[17:18] ad on meta didn't click on it but then they saw this ad on Tik Tok and didn't click on

[17:22] it and then they did a branded Google search and whatever that may be helpful for the few brands that

[17:32] only advertise on Google Meta Tik Tok and that's it and it's just there's a big number of those though

[17:41] small number sure like you know small to midcap you know companies that are still made you know still still

[17:48] spending millions of dollars in ads each year and and and they need a solution and they maybe they don't

[17:53] want to go to a a rocker box or a triple whale or North Beam or whatever and so this

[17:57] could be potentially helpful for them right if you're only spending on Google Facebook Tik Tok you don't need MM

[18:05] right that's that's overkill at that point right yeah yeah but to once you get beyond that when you have

[18:12] five or six channels and you include influencers and podcasts and TV and radio and CTV and all these other

[18:19] things then it's like this is this is just distracting and wrong and it's not going to help you and

[18:25] it's probably going to point you in the wrong direction because you're not capturing the entire customer journey you know

[18:31] the mythical entire customer journey yes right you're only you're searching for your keys under the lamp post light even

[18:37] though your keys are actually in the drain under the dark exactly um so it's kind of back to that

[18:42] whole thing it's like okay I mean I could see a scenario for some certain clients where this could be

[18:47] helpful but it it confuses me that Google is continuing to put as much resources and effort behind this Yeah

[18:57] as they are versus you know maybe putting more resources under into Meridian or into I mean I think they

[19:07] are putting resources into Meridian so I I don't know if it's a either or so much as that it's

[19:10] just a um a disconnected uh path or or perhaps one that is being requested by the loudest voices in

[19:18] the room potentially i look I I I always have deep affection for Google Analytics as I know you do

[19:24] too in terms of how what it's done for our careers but it's one of those things that I just

[19:26] don't think it's been the same platform and obviously hasn't stop the same platform since J4 but even then when

[19:31] you think about their core development methodology or things that they're bringing to market they really haven't been the same

[19:36] since um well I guess the pre- Paul Pelman era like when when he came in with what was the

[19:42] name of it the Paul Pelman um multiouch attribution thing when they tried to start Google attribution adomery adometry yeah

[19:48] yeah when when they acquired kind of dometry cuz like we're going all in on attribution that and then they

[19:52] sort of abandoned after a little while cuz they figured like oh we don't have connections we can't do x

[19:56] y and z i think that was a moment where um the the core function of the platform really diverged

[20:02] from um web intelligence toward being more of an ad enablement platform and they're really focused on how do we

[20:08] influence ads with this data and that's what I think you see from from like the Google signal and and

[20:12] how and how that's all encapsulated that's not a bad thing it's just a different thing it's it it's not

[20:17] necessarily serving the original purpose that it was and I think it's to a degree I will always complain about

[20:20] that but I also go I get it you are here you've given us the biggest free analytics platform of

[20:26] the entire world ever and if we want something different we got to go and build it and buy it

[20:31] uh and that's why Adobe exists uh that's it's why Pwick exists right like there are these players that do

[20:37] existing ecosystem to to be mindful of like this is this is our goal but it's not I think what

[20:41] Google is so I thought that was particularly interesting sure now one thing I had a question about so like

[20:45] we've talked about Google Analytics and the the stuff that they're putting in there kind of continued focus on MTA

[20:51] and I remember I thought I saw something around um like scenario planning within Google Analytics am I am I

[20:57] right on that or was that something that they Yeah there was also scenario planning with Meridian yeah so is

[21:04] there are they like doing trying to do both of these or am I did I miss that i think

[21:08] there was some illusion to perhaps not explicit statement of and so look the the nature of any announcement that

[21:14] comes out of GML is always a that's interesting let's see what happens six months from now and if six

[21:21] months from now is there's you know the validation you start to see these tests coming out then yes I

[21:25] don't know how you scenario planning in that environment I I attribution is inherently a retroactive exercise so in order

[21:30] to do scenario planning you have to have some kind of forecasting capability which I what kind of forecasting capability

[21:35] you have in in in there unless building a mini mm in in that environment maybe they are i doubt

[21:40] they are uh but I I did not see that but to the main point that scenario plan this was

[21:47] I think one of the the the very big announcements um truthfully when I went to the um the demo

[21:53] desk uh I asked to see the scenario planner actually did show me a PDF um which and this is

[21:58] one for Meridian right so it's like that's right like a standalone application that is sort of interactive dashboard that

[22:05] you can plan like do what if scenario planning what if I spent here what if I spent there what

[22:10] does it forecast yeah exactly it it is it is sort of delivering on the core promise of mediumx modeling

[22:16] to be the the the the enablement planning vehicle for your business and exactly what they said is um we're

[22:22] integrating um we're integrating Meridian with Meridian scenario planner for interactive scenario planning um capabilities for budget allocation directly in

[22:31] a sharable dashboard um so this is going to remove the need uh to interact with the meridian code so

[22:36] obviously right now if you gives you I think a PDF print out um running for running different uh optimization

[22:42] scenarios for uh the budget planning process after the model is finalized uh they said it's going to be available

[22:48] later this year so between now and December 31st I would imagine that they're going to be targeting preQ4 because

[22:55] they want they're going to want to get ahead of holiday budgets for sure although with tariffs this year like

[23:00] I'm getting a lot of questions about like how should we be planning for Q4 i'm like I don't know

[23:03] how I don't know how I'm planning my own life for Q4 at this moment of time let alone what

[23:06] the industry needs to be doing i've already told my daughter she's getting 12 less dolls exactly yeah she only

[23:12] needs one one each uh you know be the American Girl doll exactly i'm telling my kids they have to

[23:18] give a doll back so they're getting negative dolls no uh but by using Google to say by using the

[23:26] Meridian scenario planner teams can quickly run multiple what-if scenarios to optimize their strategies all in a user friendly dashboard

[23:32] to easily visualize results um making it easy to glean learnings from your MM to share with stakeholders which um

[23:38] does sound lovely it does i think it's it's it is what we all want actually and frankly it's what

[23:43] the a lot of the value of being a MM consultant is or you know it's but the the challenge

[23:48] I immediately saw with this I'm like oh gosh mm is so opinionated uh like the the the I have

[23:54] I've been doing this speech recently where I talk about the nature of AI I've gone to a few conferences

[23:59] done it recently and one of the things I talk about is machines learn humans experience and I kind of

[24:04] just stumbled into that phrase but a lot of people like oh yeah I was like that actually is probably

[24:08] such a good allegory It's such a good scenario of how we should think about MM because they are just

[24:14] it's just machine learning that's happening and it's not directly tied to any experiential scenarios or or frankly I don't

[24:20] know Vibe MM which is maybe the future of look Vibe MM's coming next it wasn't on my 2025 big

[24:26] death but that's what's happening is Vibe MM you're triggering me i I just saw someone on LinkedIn a couple

[24:31] days ago talk about they were talking about MM and just just ask Claude to do it totally oh I

[24:37] saw that i saw that yeah go ahead have fun look and you can do it and it will do

[24:43] it it's just the question of how much are you going to trust that just like I will say Claude

[24:48] do my taxes or you know I don't know if I trust it right now and it depends on the

[24:53] degree of certainty you require in these scenarios where it's like if if I'm just doing an MM for fun

[24:58] which maybe you do MM for fun I don't know but if a fun all right it's fun to look

[25:03] at just like a lot of the way I still use um LLMs as for fun to see can I

[25:07] create a video of this happening u by the way VO3 holy moly that's wild it's going to destroy the

[25:14] influencer ecosystem like everything's going be possible I've got a lot of thoughts on that it's not a measure up

[25:18] topic but I'm I'm just going to be talking a lot about that randomly um but it is this this

[25:22] this moment where I think they they want to make mm accessible they want to make it uh attractive and

[25:29] engaging to a non-technical non statistically enabled marketer but I think it's at a disservice just like we saw when

[25:35] they bought meridian to market there was confusion does this have an incre integrated incrementality testing tool inside of it

[25:40] no it does not um does this does does GTM mean you no longer need a developer that was the

[25:45] go to market messaging long ago and and you and you go but what we've actually seen is that you

[25:50] need twice as many developers because it's now that much more complicated that is the perfect analogy yeah Google tag

[25:56] manager now you can you don't need a developer because you can put the tags on But clearly that was

[26:01] never the case and people the boon of Simo's career was being like trust me you're going to need support

[26:09] here and I have all of this information he was like before that I was just like oh tagging is

[26:14] it's not easy but it's just arduous after that I was like tagging is so complicated and there's so many

[26:19] things that we have to think about like payload elements and like you know delayed uh variable population all these

[26:25] things I just hadn't thought about previously that now are hyper relevant yeah and and same thing on the MM

[26:30] side right like you know Michael Kaminsky from Recast always says this and I I I parrot uh this phrase

[26:37] a lot which is it's ridiculously simple to build an MM whether you're you go use Robin or Meridian you

[26:44] could you could build an MM in five minutes if you if you have the data together already like that

[26:50] that's not the hard part like building the MM is easy it's it's but is it right yeah you're like

[26:57] there are an infinite number of mathematical equations that can give you a good for you know give you a

[27:03] good model that looks like it's right matching reality but you know there's only one truth so this is a

[27:11] I think an issue of spectral validation or spectral quality and that I think there are a lot of things

[27:16] in life we acknowledge um look it's a a $2,000 fridge or a $200 fridge and you look at both

[27:22] and like they both keep food cold and you're like sure but one of them is like a one that

[27:26] goes under my desk that won't go any lower than 35° and one of them has a camera inside to

[27:31] tell me what's happening with like the due dates of my products and D and you might not need the

[27:35] one that does all the stuff with the camera but there is a reason they are priced differently and there

[27:40] is a reason why there is a spectrum of MM solutions out there in the market and we talked about

[27:45] this when we talked about Robin Meridian PMC we did Meridian 4 when we did our Meridian episode um and

[27:50] it might be I think once planning comes out it'll be really interesting to do Meridian 2 uh and and

[27:55] maybe we can even um I I know a few folks on the team over there and maybe I can

[27:58] get one of them on the show so we'll uh we'll see or if you out there if you're a

[28:02] listener and you are one of the Meridian uh uh engineers someone who's working on the stand out please come

[28:08] get nerdy with us because we would I would I would just love for that opportunity to to have that

[28:12] discussion yeah but yeah you know like on on the like back to the idea of like trying to you

[28:18] know Google's trying to make this accessible to to more people to less technical people and I kind of have

[28:24] two minds about this one is um that's great right i I I do the same thing i'm trying to

[28:31] spread the message get more and more people absolutely aware of and interested in MM um so I'm glad that

[28:38] they're trying to like bring more people to the table but at the same time I think there's there's a

[28:43] danger of of of making it too easily accessible the perception is is overly simplistic right i don't know if

[28:55] it's like that easily accessible so much as like you perceive it is that simple and therefore you're willing to

[29:00] get dangerous with it or maybe I'm not pleased yeah no exactly because like back to my point like anyone

[29:05] especially now with Chachi T and Claude and well you could you could build a model in Meridian tomorrow in

[29:10] 10 minutes but here's the danger right just because it's easy to build the model right and Google and and

[29:17] Meta are making it easy and more accessible that doesn't mean you should say okay here's the model and here's

[29:24] the output now I'm going to go change my budgets around and and watch your business implode because that's right

[29:29] didn't validate the model you didn't do any back testing or forecasting or didn't didn't hold out hold out testing

[29:34] you didn't actually validate that what the model is saying was grounded any sort of reality right which a lot

[29:41] of these open frameworks don't do right they're still lacking severely lacking in any kind of capability to validate the

[29:50] model right so with Robin you can do uh you can do time series validation and so it splits it

[29:56] into a training testing and validation data set and it it will basically do that with Meridian it randomly holds

[30:05] out dates which if you're if you know anything about time series forecasting you can't just do random sampling to

[30:13] test how it works um and and that's like that's the extent of what they offer um they don't offer

[30:20] any kind of um parameter recovery exercises like on the meridian side with Beijing processes um there there's no back

[30:29] testing or or hold out like forecasting seeing how you uh how accurate the model is on data it hasn't

[30:36] seen yet and this is something that again I'll go back to recast they are kind of leading the charge

[30:42] yeah and this very rigorous validation process across a lot of different parameters whether it's parameter recovery uh out of

[30:50] sample forecasting back testing like they're all in on this and have to be if you want to have confidence

[30:58] in your model right otherwise good luck because you're just like hoping that you got the model right and and

[31:05] hoping that it's kind of like going to point you in the right direction but you you're you're flying blind

[31:10] but I I I as as you're talking I'm like God isn't some of this just a reaction to the

[31:14] fact that we haven't made it easy it's not easy but there are avenues to explain it in in in

[31:22] perhaps more relatable ways and and and for better or worse there is a little bit of value in it

[31:28] being a really complicated thing because that is often how you convince someone that you can't do it on your

[31:33] own you need me to help you with it or you need like there's there is a sort of uh

[31:37] um exclusion like we're going to make this sound complicated just like my plumber when he comes over he's like

[31:42] "Oh this is a pretty complicated problem." And I don't like I don't know enough about plumbing to know that

[31:45] but if someone comes over to like "Nope it's not that simple it's not that hard at all in fact

[31:49] we'll tell you how to do it over the phone." Great but I just didn't pay you $400 um and

[31:54] that becomes now a question as to um what is the val like is there value uh in in making

[32:00] it more explainable or or making it easy to understand maybe I think there is probably and all those scenarios

[32:05] I'm like well we got to do a better job explaining those and then there's this other question that pops

[32:09] into my mind and that is but don't you think in this era of like AI can do everything and

[32:14] I'm not an AI absolutist but I am like an AI optimist like I think there's a number of optimist

[32:19] and in in sort of business process I'm a like long term I'm terrified of what's going to happen but

[32:24] um it's a two things are true at once for me there is a future where the AI or an

[32:31] agent essentially can be your mm explainer and you can just say these are things that I'm thinking about how

[32:37] would this work and they will perform the back testing they will do right like is that a is that

[32:42] a viable scenario and this is just like look we're going to get you to come to the water and

[32:45] then we're going to put those features in and we're actually going to make this for real for real but

[32:48] we need a core mass of people who want to be in yeah i mean I completely I'm I'm with

[32:56] you i'm like an AI optimist um and my position is my my view on this has kind of changed

[33:01] over the past six months i used to be like six months ago I would say there's no way like

[33:07] Mm is so complicated and it's not it's not um there's so many nuances to it and it's you know

[33:15] you have to have knowledge about how marketing works and it's different from client to client like I used to

[33:21] think it would be that that'd be too hard to do but I kept it in my mind like open

[33:26] like well yeah but if you told me 10 years ago that we would have chappie GBT or 03 search

[33:32] 10 years ago I'd be like there's no way that's possible it's too complicatedly too nuanced and so like even

[33:38] 6 months ago I was like maybe there's a chance like in the future yeah it's progressing rapidly maybe that

[33:43] could happen and and now I'm more bullish on that i I think that is completely plausible and and maybe

[33:50] even probable within the next couple of years i was going to say every time I throw out a number

[33:56] and I go next couple years it's like no that's happening in the next six months yeah yeah and you're

[34:00] just like okay apparently it is and and it's a I don't want to say there's going to be a

[34:06] crisis amongst measurement practitioners but there is a real question as to what is your value in an ecosystem where

[34:11] you do have an agentic layer on top of MM it's not just assisting with a readout but it's also

[34:17] performing the the validation analysis and and and ensuring you know degrees of accuracy and and there wasn't an incrementality

[34:24] test design in when it came out but why not like that seems like a logical play right well and

[34:29] there so I would split into two two buckets right there's the there's the part that AI could very easily

[34:35] handle right validation of models um statistical measures to to see is this model actually uh capturing the underlying reality

[34:43] is it you know are we capturing the causal nature of of marketing on sales things like that like the

[34:48] you know uh parameter recovery exercises the out of sample forecasting the hold out test things like that like that's

[34:55] mechanical and and easy for AI to to do but the part that I think it very well will get

[35:04] to the point where it can do but it's less of the mechanical more code related like automate this process

[35:11] which is like validation test it's a process You can automate it it's code based the other part is talking

[35:17] with the client to say "Okay what what marketing channels are you using?" And they give you a list and

[35:22] then you you build the model and you realize "Huh there's this weird spike in the data that doesn't seem

[35:26] to be explained by any of the channels they told me they were using let me go back to the

[35:30] client and say "Do you happen to be using email or SMS marketing because I see these regular spikes?" And

[35:35] they like and the client says "Why yes I we are doing regular email promotions of 20% off." and and

[35:42] the the model has to say "You stupid idiot why didn't you tell me that beginning?" Except the model's more

[35:47] polite and says "Oh thank you so much for sharing that with me that'll help me build a better model."

[35:52] They are British i often think of I I hear that though and I go "Okay so there's a there's

[35:59] like a limit of information that we're providing right now." But there is also a feature state where now you've

[36:03] got your marketing plans on your Google Drive and you just say "Give me access to everything." Uh and you

[36:08] can know in that cord situation where it tries to what was Anthropic had that LLM that they tried to

[36:13] shut down like it was an experiment so it wasn't real but like they gave it access to the engineers

[36:17] email account and found out that they were having an affair and then they and then they were like I'm

[36:22] going to expose your affair to the world if you're trying to shut me down this is like I'm fighting

[36:25] for my life and you look at that and you're like well that is the literally the like this is

[36:28] Terminator uh but but I I I go "Yeah okay so that's just like an information sufficiency problem." And there

[36:36] is a world in which that can be accounted for early on especially if you're seeing like if if it

[36:41] exposes itself in the data like weird little dips and stuff where you go like we've seen this before this

[36:46] usually means you haven't told us everything about the data or um you know we already from Jump have access

[36:51] to all of the like we got access to Google Analytics so we we know what traffic is coming through

[36:56] on on site and so we know that there's email coming through and maybe you didn't integrate and you're like

[37:00] okay well we still know this is coming from you know uh an email operating system whatever like there are

[37:06] ways to figure these things out in the log records that if you've got access to the logs on a

[37:10] website you can kind of retroactively figure out what is the marketing mix of this company outside of what just

[37:15] the paid media you're exposed to is true right and I guess that goes back to sort of a data

[37:19] quality issue right is is want to like set up correctly are the UTM parameters tracking all the things that

[37:24] they're doing correctly and uh which is also something AI will come in and solve for figure that out right

[37:32] so you know like we did it again we we went down the MM rabbit hole we did i mean

[37:38] that's what this that's what the podcast should be called is the MM rabbit hole all paths lead back directly

[37:43] to MM at all times right but uh so maybe we can take a step back back to Google Marketing

[37:49] Live yes what were some of the big announcements that we haven't talked about like obviously AI and search was

[37:55] was huge um so yeah AI and search is is just this thing that I think is we're we're not

[38:05] ready for it i think there have been a lot of jokes around um AI overviews for a couple years

[38:10] now since they well since at least a year there was only one year ago by the way that they

[38:13] said we're bringing AI views to everyone and that was at Google IO so a year and change um and

[38:19] they were like "Oh rocks on pizza can I eat rocks on pizza?" I'm like "Yes a couple rocks a

[38:23] day are fine for you." Or I found one a while back was "How many cigarettes can pregnant women smoke?"

[38:30] Like up to recommend up to two cigarettes a day i'm like "What that's okay." But I think we're you

[38:34] know not that we're fully past that and if you see like Lily Ray who's like big in the the

[38:37] world she'll show you things like if you ask it is it is it 2025 it'll be like "No it's

[38:41] not 2025." And I'm like "Oh come on that's it's not a hard one just like look look at your

[38:46] date i know you have an internal clock Google um but but but but the way we are searching is

[38:51] changing pretty dramatically and with the launch of AI mode in particular now it's been a feature in Google um

[38:57] labs for a little while but AI mode is the first um sort of real progression toward an agent companion

[39:04] experience that we've seen from Google and and that's why earlier the the very top I said search may be

[39:09] even too reductive of a term and Google was even championing this idea of its discovery and I I honestly

[39:14] I I I lean into that very hard because I do believe that um the nature of how I've searched

[39:19] historically has been like I will Google something and then I will look at the front page i'll look at

[39:23] the you know the template link and I'll click on six of them and open them in six different tabs

[39:26] and I'll click through them because I didn't have a good search experience because it didn't match my query and

[39:31] frankly I'm an uninformed searcher i the reason I'm searching for something is because I need more information about it

[39:36] so I may not even be framing up my query in the right way and what this is going to

[39:39] lead to is a lot of these divergent paths that folks may go down in terms of how they're searching

[39:43] and and how that's changing what does that ad experience look like we don't know i think it's going to

[39:47] be a particularly difficult measurement challenge because they also announced AIAX for search which is the way that Google is

[39:53] now thinking about their ad product for search ads in particular um one of the things this is different than

[39:58] PMAX right separate AI amazingly yeah aix search uh I know it's it's one of those things you're just like

[40:06] well I there are so many words here and I know what each of those words mean alone but what

[40:10] do they mean together um different from HBO Max i'mve I've been told yeah which isn't a thing anymore i

[40:15] think they No no it is no Max yeah like it's Bamble now it's Constantinople uh oh god someone should

[40:25] do a parody of that but anyway the um it is different and and and but how is it different

[40:29] we don't quite know yet because PMAX right now is you performance max is um integrated of all the different

[40:36] types of inventory that are available in the Google ecosystem um we did that episode with the VA Hopkins um

[40:41] by the way and she's got some really exciting news coming soon so we may get her back on the

[40:44] show to talk about some things in her new career i can't say too much more uh that's down to

[40:48] my NBA uh but the but the um the thing with PMX is it did search text ads it did

[40:55] video it did display um and you know Yeah all of it if it's in the Google ecosystem it's part

[41:01] of PMX and they would optimize the best ad to show the best time to the best person it's a

[41:04] good theory um AIS for search we're not entirely clear where the line begins and ends with PMAX because if

[41:11] you've got search text ad assets in a in a PMX campaign what's the overlap with AMX for search we

[41:18] don't know um I think they're going to diverge a little bit more i think we will see what this

[41:21] looks like but it might just be a future state where they're like "Yeah we want people to run demand

[41:24] gen and we want people to run AMX for search." don't know they would change it up because I actually

[41:28] think PMX was sort of hitting its stride uh but they said some of the results were pretty good for

[41:32] some of the advertisers they were working with so the jury remains out but I think it's a new measurement

[41:36] problem we have to reckon with and they do have uh now um asset level reporting and so that's I

[41:42] think can be a particularly interesting thing for how can that be leveraged by the measurement community as we think

[41:46] about here's the thing even if you can measure it can you influence it and the answer is no right

[41:52] because like what do you do in that environment you go unless you see if PMAX is all you like

[41:57] dumping a bunch of your budget into display cheap ridden display i don't you can't opt out right yeah you

[42:07] can't change it's going to happen either way the only things in theory you could do would be to put

[42:10] some negatives on top but negatives have this inherently negative or negative they have um negatives present a problem for

[42:17] how the machine learning works right they they they they sort of create these artificial constraints that it's like if

[42:22] I let you I'm like I'm going to teach you German but you're not allowed to use the word duh

[42:25] and it's like god how are they going to learn half the genders of the languages and you're like you

[42:29] just won't be able to but you got to work it out somehow that that word exists that's kind of

[42:34] what you're asking them to do right and so that's why they're like go fully open go everything and and

[42:38] so that's sort of the been the mantra i don't know what that means um I I I think that

[42:42] is a particular interesting one i think the other one that's really interesting is um Veo now this was a

[42:46] Google IO announcement in particular but Veo 2 if you played around with it was um really pretty powerful right

[42:52] v2 allowed you to create 8 seconds video 8second videos within um Gemini based on a tech prompt to um

[42:58] a visual um scenario and I've made several of these cuz I think it's pretty interesting they do have some

[43:03] safeguards in there but we're seeing a lot of memes now going around the internet where I'm like there were

[43:06] no safeguards applied like this is this is this is like I saw one yesterday of a guy jumping into

[43:11] a volcano now i don't know if that was Veo though right because there's a lot of these other uh

[43:17] visual platforms out there the reality is it is changing what we know as reality and I don't know if

[43:22] you can look at reality anymore through the lens of what you've seen on a screen um so much as

[43:26] only if you are exposed to it with your eyeballs directly can you possibly validate that which is very interesting

[43:32] um but Beo 3 now allows for um actual audio as well so you can like code language and you

[43:38] can you can have make a little show and one of the things that they showed in the afternoon was

[43:43] the Google Super Bowl ad from last year and this was not on the live stream out there this is

[43:47] insider info um they they had the the the Super Bowl ad um they they had done they spent millions

[43:53] you you spend a lot of money on the creative for a Super Bowl ad because you're spending a lot

[43:56] of money on the placement but in this particular one they had uh it was like the Americana shot of

[44:00] just like here's people in a tavern here's people in a school here's someone making a guitar and they're like

[44:05] Google we're here for you and it was a really nice sweet heartwarming ad um then they recreated that with

[44:11] Veo and it was like shot forshot recreated and they're like this cost us $400 this cost a million and

[44:17] I'm like wow that the power of Veo or VO I don't know I can't remember exactly how it's pronounced

[44:24] but the power of this platform to change the way that creative is developed change the way that storyboarding works

[44:29] in these environments heck even to change the way that we think about datadriven creative when I think about this

[44:34] as a measurement problem I gosh imagine if you can iterate on this ad in these different ways that allows

[44:40] you to take in some of this data feedback of like where people are either you know tuning out or

[44:44] skipping your ad at that point and remix that particular element of it without going back into the field and

[44:48] saying we got to get more shots of the guy with the guitar because everyone loves the guy with the

[44:51] guitar like you know that's amazing or you you you partner up with a a a vendor like system one

[44:57] that does creative testing and instead of like you know paying a lot of money to have the shot and

[45:03] then going and testing it and realizing it doesn't perform now we have to money like it's it's just a

[45:08] lot cheaper to to do this yeah well and when I think about you know I thought it was a

[45:14] really interesting justosition to see what Google was putting out there and I one of my a good friend a

[45:19] good industry friend is this guy Naheen um he he said this is my view has changed on this my

[45:25] view is now I there was a period where like would make m or creatives redundant it it is now

[45:30] that I view this as the clay that they are modeling with and I thought that was a really apt

[45:34] um visual as to what this medium is now and how it's going to work for for marketers for iteration

[45:41] testing all all these things that they can be doing um but the creative is only as good as what

[45:45] you make it so you still got to have these good ideas um to support you in coming up with

[45:49] those ideas but it's still a very human in the loop process and I thought the justosition between what Google

[45:54] was presenting there and what Mark Zuckerberg has been saying on the Facebook earnings calls when he goes and says

[45:59] just give us your money tell us how much you want to pay and we'll do the rest you don't

[46:02] need anyone else in there and any other function and then Sam Alton's like in agencies 90% of the things

[46:07] you do are useless we'll replace it all and like both of those are really um pessimistic takes on what

[46:13] the role of an agent or what the role of a human is in this process and I thought that

[46:16] Google look a lot of folks have issues with Google i I I personally sort of always am a little

[46:21] bit more pro Google but that's that that's a bit of a bias I have but in this particular instance

[46:26] and I'm aware that bias but in this I'm like Google is doing it right this is this is the

[46:30] best way to do it um in this world but there's still brand safety problems there's still going to be

[46:34] problems associated with um the replication of celebrities and I've been running some tests on various see can I get

[46:39] a celebrity replicated can't do it on Vea can't do it on Gemini can't do it on JBD can do

[46:43] it on Llama llama is a wild west man they will let you use anything on that platform why does

[46:48] that not surprise me right exactly yeah so and that kind of speaks to Meta's lack of guard rails perhaps

[46:55] at times um I think they're doing some incredibly powerful stuff over at Meta don't think it is um the

[47:01] way that these things are going to operate in the future as to how um creative will be curated um

[47:06] especially because brands want to voice this right brands don't want to have this thisization um anyway I know we're

[47:11] getting a little bit off the measure uh topic but all of it does come back to measure because at

[47:15] the end of the day all the more granular and specific you get with the attributes that you're able to

[47:20] test and and and and you know iterate on the more that measurement wills have the the the primary seat

[47:26] at the table as to understanding what we do Yeah which makes me bullish on measurement so when I say

[47:31] are we getting replaced by LLMs no but our roles are going to drastically change 100% 100% u well hey

[47:38] before we go because I I know if you've been if you've listened to us the whole time that's awesome

[47:42] and we we thank you so much um and you may have seen this on our LinkedIn but if not

[47:46] and we'll put the note we'll put the uh uh we'll put a a thing in the in the show

[47:49] notes um we have these measure up stickers all right uh Jim was Jim is the the chief merchandise officer

[47:58] of the Measure Up show uh and he went out and got these i think he gave some shirts out

[48:03] at um at uh uh the Measure Camp where we did the where Jim did the live show i wasn't

[48:08] I was there in spirit uh but but that has got a lot of people love that people you know

[48:12] people are interested but we're giving these out for free and the reason is is because we want your laptops

[48:17] to look really cool um and we also want you to share it with a friend so if you do

[48:20] want one uh we'll put the link in there there's just a quick Google form we won't share or sell

[48:24] your information but you too can be rocking a measure up sticker uh and look as cool as myself and

[48:29] Jim uh which is not that hard to do it's a low bar low bar low bar exactly uh but

[48:37] no it's been fun well Simon you we've reached that time where people are at the end of the podcast

[48:43] and they're asking themselves you know I wasn't at Google Marketing Live and I'm not sure what to do but

[48:50] uh you know I've listened to to Jim and Simon and you know they they tell me that all I

[48:55] can do is just keep measuring up