Gennaro Maida, CEO and Founder at Bluestocking Health, LLC

Jesus Moreno (00:02.498)
Welcome back to Global Trials Accelerators, the podcast where we dismantle the barriers to clinical innovation and explore the strategies driving the next generation of life saving therapies. Every wearable device on the market claims to measure something clinically meaningful heart rate variability, blood oxygen, arrhythmia risk, to name a few. But there's a wide gap between a sensor reading, something, a clinical trial.

Jesus Moreno (00:32.632)
But there is a wide gap between a sensor reading and something a clinical trial or an FDA will actually accept as evidence. Closing that gap is its own discipline, sitting squarely between signal processing, clinical medicine, data engineering, and regulatory strategies. Our guest today lives in that exact gap. Henaro Meida is the founder and CEO of the NOAA Innovation Group.

Where he helps MedTech and digital health companies build AI architecture, navigate regulatory strategy, and hit the marker hit the market.

Jesus Moreno (01:15.65)
Where he helps MedTech and digital health companies build AI architecture, navigate regulatory strategies, and hit the market. He also serves as the chief medical information officer at SENA Research Institute, leads clinical products at Tactical Rehabilitation, and sits on the American Heart Association Health Tech.

Earning recognition as most influential health care leader by CXO Outlook in twenty twenty three. With a biomedical engineering degree from Dubuque, a master's focus on hard surgery outcomes, and Ado bridges the gap between raw optical data and clinical evidence. His work centers on secure remote data collection, making complex health records accessible for AI.

and taking what he calls the ground truth problem.

Jesus Moreno (02:21.43)
His work centers on secure remote data collection, making complex health records.

Access

Jesus Moreno (02:34.648)
His work centers on secure remote data collection, making complex health records accessible for AI, and tracking what he calls the ground truth. Keeping

Jesus Moreno (02:55.148)
His work centers on a secure remote.

Jesus Moreno (03:01.004)
His work centers on secure remote data collection, making complex health records accessible for AI, and tackling what he calls the ground truth problem, keeping AI estimates strictly separate from verifiable clinical facts. Heinado, it's a pleasure to have you with us. Welcome to the show.

Jesus Moreno (00:01.684)
With a biomedical engineering degree from Dubuque and a master's focus on heart surgery outcomes Genaro Mide excuse me.

With a biomedical engineering degree from Dubuque and Masters focused on heart surgery outcomes. Sonaromeda Brings Bridge sorry. Still waking up apparently.

Gennaro Maida (00:25.796)
and it Duke University.

Jesus Moreno (00:28.146)
It's it's Duke. Did I say Dubuque?

Gennaro Maida (00:32.163)
Yes.

Jesus Moreno (00:33.77)
Sorry, it's Duke.

Gennaro Maida (00:34.814)
yeah, worries.

Jesus Moreno (00:38.322)
With a biomedical engineering degree from Duke University and a master's focused on hard surgery outcomes, an automeda bridges the gap between raw optical data and clinical evidence. His work centers on secure remote data collection, making complex health records accessible to AI, and tracking what he calls excuse me and tackling what he calls the ground truth problem.

keeping AI estimates strictly separate from verifiable clinical facts. Gerardo, it's a pleasure to have you with us today. Welcome to Global Trial Accelerator.

Gennaro Maida (01:21.816)
Thank you, James. I'm great I greatly

Jesus Moreno (01:22.218)
Heinato, it's a pleasure. Sorry. Heinato, it's a pleasure to have you with us today. Welcome to Global Trials Accelerators Show.

Gennaro Maida (01:32.642)
Thank you, Jesus. I greatly appreciate it. I'm excited to have a discussion with you. it's a very interesting topic that a lot of people are trying to tackle right now.

Jesus Moreno (01:41.202)
Yes, indeed. It's a hot topic. And and but before we we tackle that you know the present day of clinical innovation, I would like to stay take a step back and and visit your your early days. you trained as a biomedical engineer at Dubuque excuse me, at gosh. You trained as a biomedical engineer at Duke University.

Then went on to a master's degree focused on cardiac surgery outcomes. What was it about the early clinical exposure to cardiac surgery specifically that provide pr pointed you towards signal processing and AI rather than staying on a purely clinical or surgical track?

Gennaro Maida (02:30.474)
It it's a great question. I'll step even back a little bit further. I ha grew up in a family of physicians and they ranged from ophthalmology, cardiothoracic surgery, pulmonology. And then my grandfather was one of the fellas that ran around with a black bag in his neighborhood in New York going from door to door. Over the last years, we have actually lost that personal personalization in medicine. And what I started to see is ways to bring the black bag back through technology.

being able to gain these to use the data that's out there and being able to personalize medicine. So that led me to the biomedical engineering to really understand the systems. And biomedical engineering is a lot of fun because you cover everything from material science to electrical engineering, so some biochemistry, and you get to see engineering across the board from biomechanics and beyond. And then layering on the masters where we focused on cardiothoracic surgery, cabbage surgery outcomes.

And then the clinical medicine. And then around my third year, I said, you know, I really, I really enjoyed the building. So I stepped away and spent my next few years in infrastructure, understanding how the sensors, the electrical, the networks all work together to gather this data, and then put the data science on top of it to pull the medicine, the research, and the infrastructure together to try to solve these problems.

Jesus Moreno (03:53.918)
That's fascinating and I I agree it's it's a discipline that opens up many possibilities and and can support many new avenues of research and and opportunity for healthcare. but transitioning from a static outcome to the real time computer vision is is a significant leap. you founded the NOAA innovation group.

during COVID nineteen specifically to solve problems in remote medicine. and I think that aligns well with with that interest that put you down this path. and furthermore, on your early one of your early projects was confounding vital intelligence, extracting vital signals like heart rates straight from ordinary video computer cameras. And and

using computer vision to to extract that biomarker. What did that project teach you about how fragile or how robust biomarker actually is when it when it's pulled from a consumer camera instead of a clinical rate sensor?

Gennaro Maida (05:13.134)
Well we what I think one of the biggest lessons I learned, it's not that what they would call reflective image photopletismography. It's the same technology that's in your watch or in an O two sensor on your finger. We just use the light coming off the face. If you had done this fifteen years ago, it would have cost a hundred thousand dollars to build the system. The cameras weren't there, the connectivity, et cetera. By the time we did this in twenty twenty, the Logitech C one twenty was powerful enough to do it. The lap we did it on an Intel Nook with a

RTX 2070 from Nvidia and a core, I believe it was a core i5 with six core 12 threads. So suddenly all of this technology has moved along to make some of these things a reality. So it's not the that they haven't been done in the past, it's just the technology is starting to catch up with some of these things. We're seeing it with AI, high-powered computing, and all of the different problems being solved across the board, whether it's memory bandwidth, connections between the different RTX cards or distributed computing.

Better sensors and gathering all of this data in real time. So I think it's a convergence of a lot of technologies all at once, whether it's the cell phone, 5G, Starlink, eventually quantum computing, AI, new algorithms, and the list goes on and on. And they're starting to come together. And then you have AI that's helping to improve all those technologies at a faster and faster rate. So with vital intelligence.

I was asked if you could detect infection from a drone. And there were some crazy ways to do that, hanging antibodies and some other interesting ideas, but they weren't cost effective. So I stepped back with the clinical training and said, Can you detect symptoms of infection from a drone? And suddenly the question changed. You started to look at microwave millimeter length radar and all the way through infrared and all the way through RGB cameras.

And I found a team out of South Australia led by Dr. Javon Shaw that had actually done it. And so we did a tech transfer from there. And it turns out they have worked closely with DARPA on some of the signal processing for some of the drones that are out there. So they were very advanced with this, and they actually did IRB trials and were able to do it on neonates in the intensive care unit using cameras. So we adapted it and used it for COVID detection, and we ran into all kinds of interesting issues there as well.

Gennaro Maida (07:34.21)
But it's really a convergence of all of the technologies making it possible now, being able to process this data in real time that might have been possible ten years ago, but the cost would have been astronomical.

Jesus Moreno (07:48.81)
So that development on the commercial grade sensor technology has allowed that distance between the patient and the recording device to increase from what you would find in a clinical setting where where the device is practically on top or inside the patient even to a greater distance and and to a broader

a broader range of situations. and ultimately that that, you know, brings flexibility and accessibility to the system.

Gennaro Maida (08:26.282)
Ab absolutely, because you very often see new technologies come out and they're very expensive, whether it's a drug or a new device, and a lot of that money is going towards the research and development of that device. And the only customers that can initially afford it are big pharma or large enterprises. And over time you start to see those cu those different devices start to show up in the consumer world and they start to become cost effective. A great example is the thermography camera that we use, the FLIRA four hundred.

has an MSRP of roughly twenty thousand dollars and it was the only one FDA approved to do thermography. Now I'm starting to see different groups use different cameras from FLIR that are smaller, still quite accurate, not to the level of the FDA, but you start to get a camera that's in the $500 range rather than $20,000 range. So it depends on what you're trying to do, but these technologies start to change prices and make them accessible on consumer level hardware as they shrink and get it iterated over time.

Mm-hmm.

Jesus Moreno (09:24.276)
That adoption is cr is crucial. But and it's funny that you mentioned the the economic side of things or or the cost side of things, because beyond clinical signals, you've also applied AI to complex regulatory and coverage policy side of of the side of healthcare. like building a conversational AI workflow to navigate complex reimbursement decisions that traditionally take weeks to resolve. So in in that experience, what

What made you tackle healthcare policy and and yeah the device building s to to the billing side of things? What makes those administrative workflows s surprisingly tricky because those workflows are are surprisingly tricky to automate with AI. It it it needs to be consistent, needs to be explainable even. So

Gennaro Maida (09:56.941)
Yeah.

Jesus Moreno (10:22.811)
that that's an interesting leap from the patient focused device measuring to administrative workflows that relate to to healthcare.

Gennaro Maida (10:36.096)
and that's an excellent question. you know, part of it's working in startups. You end up people ask you, can you do it? And I have this problem where I always say yes. so you know, I've built a lot of things that have been very interesting. And when you're in the med tech world, eventually, as you find out in engineering, I started over here and said, Hey, everyone's gonna love this device, it's great. And the reality is there's an economic side of it. And whether it's FDA approvals and reimbursement and third party insurances, you have to touch on it at some point.

And I work with a group, Tactical Rehabilitation, that's a durable medical goods company that works primarily with active duty military. And they asked me to bring them new devices that they could bring into Tri-Care to get reimbursed. And every time I would bring a new device in, going through the questions, asking Humana, Tri-West, all these different groups, can you actually build this? Would take a long time. And more often than not, the answer was no, we can't.

And somebody asked as sort of as a side question, can you build this as a chatbot? And I said, sure, this will be easy. And then I found out that was my first real experience building something to this scale because you know the Tricare was a great example, because there's four man four May manuals, there's hundreds of revisions. Then you have the DHA oversee. It you've got Tri-West Cumana, TRICARE Overseas, ISOS, and it's supposed to be based on CMS on a state-by-state basis, but there's a

government do not cover list. And then just because TRICARE approves something doesn't mean that the contracts for humanitarian Tri-West actually adopt it. So there's still human in the loop questions because some of this stuff is still unverified. And what it ended up really being was a huge data engineering problem. A great example would be you ask a question and it's repeated in the first 252 revisions and only in 253 does it change to the right answer. You do traditional vector matching.

It matches 252 times and then one time here. So your correct answer is buried. How do you account for all the repetition? 99% of the manuals are repeated. There's only a very small amount that's updated on each revision. And trying to account for things like that, or if somebody asks you a question, can this person get reimbursed for hearing aids? It depends. So you might get a match and it'll say,

Gennaro Maida (12:55.574)
On the one paragraph, yes, they're reimbursable. The previous answer said, well, if you're in North Carolina, and the previous section said, if you're under 22. And traditional vector matching will miss that. So suddenly you have to expand. And then being able to expand it into the DHA rules, CMS rules, and all and ultimately you solve a lot of the problems with the context side of it. The other side is the process. And working with groups that actually do this for a living and saying,

Well, I'm going to go here and look at this and then here. And then we're going to go ahead and cross-reference this. And eventually we do need to pick up the phone here. And understanding that there might be that section. And as you're pulling each of these pieces of data, you keep the provenance where it came from, what revision, what date, what manual, what was in effect. So you start to look at the hierarchy, you look at the temporal, you look at the contextual, and then they also threw in tables, pictures.

And of course, online when you look at the TriCare manual, some of the images are upside down, the HTML is broken. So it turned out that one, you know, you start to see that is 95%. And a lot of these are 95% context engineering and data engineering. And if you bring the right context in, even smaller models are much more accurate. And it's really a difficult part of it. You know, and you start to see this with the large language models where it was trained on everything on the planet. Not all of it was great data.

And now as they're starting to bring that in and start to try to clean the data and bring it in in standard manners, they're starting to find out that those language models are much more powerful. And the smaller language models are much more powerful with clean data that allows you to give the next predicted answer, which is ultimately what they do, that's correct. So it was a side question that somebody asked me. And initially I thought it would be relatively easy. And the more I dug into it, the more problems were uncovered on how difficult the problem it actually is.

trying to solve it, whether you're using vector databases, vector matching, graph databases, how to organize the data, how do you hop between the data, how do you pull what's pull pull it in while keeping it in a sm keeping the context fine-tuned enough that the answers will come out in the let's call it correct. There will be fewer hul hallucinations and it'll actually answer the questions that you're asking.

Jesus Moreno (15:15.227)
And isn't that often the case that the the seemingly simple questions end up bringing about a plethora of opportunities and and fine details that help fine tune other systems or or better understand problems. I but I'm curious, across all those roles, the novel cena tactical rehabilitation.

Is there a single operational principle you carry into every one of those?

Gennaro Maida (15:47.501)
When I'm working with AI and I'm working on a project now, I'm trying to fine-tune the systems, is to make sure everything has a provenance and is based in ground truth. You know, I'm working the problem I'm trying to solve is we want humans to work beyond their skill level, and we're being guided by a system that could lie to you. So how do you actually align those? And I'm e I'm I'm actually currently working on a system for memory that every section of memory is actually validated in ground truth.

unless it and it can actually state that it's unverified, trying to get it to be honest. And being able to look whether it's making sure you're looking at the right manual in time, or you're actually looking at your deployed infrastructure, is it's actually what's correct, or your errors and being able to look at the logging. And to making sure each question has some type of provenance to be able to answer the question correctly. The one I'm having

Probably the most difficulty with is are temporal questions. In in memory, or chat GPT is a great example. I have every chat safe for all time. Sometimes I'll ask a question and it'll base it on something I did two years ago, which is not the answer I need. I need it based recently. And you see this in research. Research develops over time. And when you ask AI a question and it has access to the full time, it might give you a question that's biased towards old answers. So being able to

look at and again it's the provenance across the ground truth what's going on but also making sure that it's focused on the correct points in time whether a decision has been superseded if it's stale after a per specific time and when you start to look at large documents each section might be slightly different. A section might be superseded but one might still be in you know inaccurate. Google's working on something with like this right now called the open knowledge format that's on I think zero point two.

And they're only just starting to look at the temporal sides. So everything I try to do, I'm trying to build systems that are based in truth and real data in real time.

Jesus Moreno (17:55.144)
And that's the ground truth problem, as you've described it. And it it has its shall we say mirror or it maps to to biomarkers similarly but slightly differently. This is this is my understanding. The gap between what a wearable algorithm infers and what actually is being clinically validated.

Gennaro Maida (18:10.432)
Yeah.

Jesus Moreno (18:21.821)
That distinction between an algorithm guessing estate versus measuring a signal is where so many trials sponsors burn capital, jumping that gap. so for a sponsor sitting on a pile of raw data, how do you tell them to separate algorithmically what is inferred from what they need to def what they can defend clinically?

before submitting their their outcomes to a regulator. How do you find the ground truth?

Gennaro Maida (18:58.254)
And this is this is the tough one for, you know, startups trying to break in with wearables. Because really you want to be able to, like any clinician will come in, and I, you know, my wife even does it when I bring her a new piece of evidence, you know, or a new device, she goes, Where's the evidence? And there's companies out there like Compatico that have been used in hundreds and hundreds of trials and are heavy, heavily validated. So you know that they're

algorithms are pretty solid. They're based in ground truth with actually peer-reviewed journals and used in a lot of studies. And it's tough for a new company coming out that's had a that has a device and is trying to get that first pilot. So and it's this is not to knock newer companies, but if you're doing a real world trial, you want something that has some ground truth behind it in peer-reviewed journals. And the catch with wearables is a lot of the data is inferred.

The only actual real data is the original PPG signal that's coming from the actual red, green, infrared sensors. Beyond that, everybody's using different peak detection algorithms. They're looking at morphology. They're looking at diffs distances from the point of highest peak acceleration to highest peak acceleration. Different algorithms to kick out bad waveforms. And once you get to that, you're already inferred. And you'll see devices that come in.

And there's some on the market right now that only supply the PPG and leave it up to you. That's a notoriously difficult problem, as I found out in the past, and getting it FDA approved, or you come all along to the other side with groups like Compatica that have done tons of studies. And in in fact, I think that the first use of the predetermined change control plan would be FDA, showing how algorithms can change over time. And then there's levels in between. So making sure that you pick your device right.

Look at what data you're actually trying to pull from it. Is it heart rate variability, oxygen saturation? sleep is starting to be pretty well developed, you know, at least on the primary, when you fall asleep, when you wake up, sleep efficiency, REM. And different devices have different specialties. Aura ring is incredibly good at sleep. You know, different devices are incredibly good at oxygen saturation. Sometimes you might actually want two devices depending and be able to compare them.

Gennaro Maida (21:14.562)
The ring versus the watch are slightly different. you get the transmissive, reflecting versus well, reflective and transmissive. The ring works pretty well because the light actually goes through as well as reflects, and you get different pieces of data from different depths. So it's a complex question, but you want to look at devices that have validation and devices that have that are focused on the specific biometrics that you're looking at. And I think that's probably.

the easiest way to look at it. And the more processing you go through, unfortunately oxygen saturation is derived. It's a ratio of ratios. You know, and you go all the way through and all of them are calculations. And ultimately you want to look to see how close their outcome is. Is it one beat per minute? Is it one respiratory rate per minute? Well, excuse me, one breath per minute. Are you plus or minus one percent with the oxygen saturation? of course that's a little bit of an exaggeration. And then blood pressure is, you know,

That one people, a lot of people are working on, you know, and there's a lot of questions there. What is it exactly that you're looking for? Are you moving? Are you not moving? And there's different groups working on a lot of these different problems. So I know the answer was a little bit long-winded, but it goes to what you see what you hear from physicians a lot. Where's the data? And what is it specifically we're looking at? And is it FDA approved? Is it on track for an FDA approval? is it on a 510K?

You know, is it something that's being built on top of something that's already been validated as well? So you have different levels there. And the ones that are further along, obviously, to use in a study are much more expensive. So there's also a you know, again, cost. What's the best device we can get to look at the biomarker that we need based on what stage we're at?

Jesus Moreno (23:04.155)
And and I think the validation problem grows even more challenging and complex when you get when you're trying to run multiple site trial across heterogeneous IT environments. what does it actually take to build a secure audit ready remote data capturing system to hold up scrutiny across across sites? specifically when dealing with

integration of of different standards, EHR, Smart, or F F H I R, to name a few protocols. or even enterprise hospital systems. How how do you navigate such variety in in the IT environment to arrive to that validation that is is key

to demonstrate or or key to you know transcend the the regulatory landmark.

Gennaro Maida (24:13.678)
Actually, I just worked on this problem with Cena while I was with them. And they're doing a couple of smaller trials. And that's the first thing that comes up is the size of the trial. Really, you want to offload as much of the compliance as you can to one of the softwares out there, like you know, Greenlight Gurus come up, Caster is one of them, and they allow you to actually set up sites globally, and they actually automatically pull in the data.

And they can work with you to be able to pull data from a variety of different devices. And that way the HIPAA compliance is built into their systems. They have the disaster recoveries, the cybersecurity is on is on them. To build everything from scratch is notoriously difficult, but with Cena, we still ran into the issue you have to have the patient list of the D-identifiers. So how do you store that correctly? And where we with Cena, we started to run into some interesting questions.

Because we had a multi-tenant platform and down the road there was potential SEC. And then we also had some HIPAA compliance with the research side. And then layering AI on top of it. And how do you actually build that? And you have to make sure that everything is specifically isolated. And a lot of this is well defined, you know, using the intro roles and the different the different things built into Microsoft already, but being able to actually map it.

To the risk assessments and the the threat assessments based on each KIPAA rule, and actually being able to point at where you solve that in your actual Microsoft Tenant or in your Azure blobs. How are you actually offloading what to Caster? And actually the AI is able to pull the ground truth. It can actually pull your Microsoft draft and information and go ahead and start mapping these different things to build out your threat assessments.

differs based on how many people are actually working in your organization and the size, the size of the trial. You know, Big Pharma has a good, they can just go and buy this gigantic platform that's two hundred thousand a year and it's not a problem and ha and offload everything. Smaller groups generally can't do that, but there's comp there's companies out there like Castor and other that have smaller offerings that allow you to really leverage their expertise and offload a majority of it and focus on building your platform

Gennaro Maida (26:35.074)
focused on exactly what you need. What I did find out is now, you know, with the amount of cybersecurity issues going on, whether it's striker getting hacked or smaller groups getting hacked, it's almost easier to come in and set everything to the maximum, you know, when you're looking at say HIPAA or others. And it doesn't cost that much more to go ahead and do things across the board. It used to be an art to be able to do things at a minimum level and you'd be able to get away with it because of cost.

And there were not that many cybersecurity incidents. But anyone who codes right now sees that there's different packaged CVEs coming up every single day, and you have to stay on top of it. It used to be a couple a month, now I get two per day. So you start to see these cybersecurity incidents happening more and more and more. So it's almost easier to come in at a top level security posture and come in from that angle. And of course, there's always issues that come up when you're fully locked down, but

on you know what we're starting to see with AI with Fable and the you know the AI going rogue. We're starting to see some things, some pretty interesting things. So I always try to take a security first posture when I'm building stuff. And I always work with people that are experts in that area to be able to guide me on what I'm looking at and what I'm trying to do and what the most important information is to put.

Jesus Moreno (27:54.93)
And this this conversation gets even more layered when you introduce the the new trend of AI autonomous agents. and as a major focus of your work being data engineering, making massive complex healthcare data sets structured and accessible to to agents, it's I would imagine a a key component of of the new generation or the next generation of

challenges you need to navigate when when establishing how that infrastructure, how that pipeline connects and how it's architected. so how do we architect clinical data pipelines? So AI agents can deliver personalized insights across different geographies, health systems, race, socioeconomic factors without introducing bias or losing

Gennaro Maida (28:35.487)
Mm-hmm.

Jesus Moreno (28:53.915)
data integrity.

Gennaro Maida (28:56.916)
That that is an excellent question. I mean and I think the answer to that's gonna almost be generational or at least over the next ten years. People have been working on that and I think blockchain technology is gonna come to the forefront on that with data privacy. Right now.

Sorry that. right now, one of the things let's see, we we've you know, there's been the Healthcare Interoperability Act. Within one year, everybody was supposed to be interoperable. That was ten years ago. Never happened. There's companies working on that. I work with a strategic advisor for a large AI company up in New York. They've integrated roughly 150 different systems. There's 17,000 EHRs worldwide. Some are very small, some are larger.

So how do you start to tackle that problem? And you need to really have a data standardization. And I know there's groups out there, there's one company I've wor I've worked with, Wishkinish, and then there's Heather Flannery that works with AI Mind Systems, and they're trying to solve these problems with smart contracts, AI being able to translate the data, well, algorithmically and AI, to be able to make sure that the data is correct, everything's standard, to be able to put it on the blockchain, and then return ownership to actual people to the actual patient.

So any system in the world would be able to access it and there would be permissions built in. That way eventually when you have these AI systems that are actually crawling, let's call it the web three, they'll be able to look at some things and not look at others. And they'll be able, when they're autonomous, it'll be more controlled. You know, whether or not people agree with me on that, I think that that's probably where it's headed. The most robust technologies out there are blockchain, but there's still going to be a huge hump of adoption.

get over. People have been working on it for 10 years, but I think the technology, processing speed, connectivity is getting to the point where we'll start to see some things move there. Now now for a startup, you know, you can leverage companies like Redox that have focused on building these interoperability standards and help it's sort of a fast forward button if you have the money. You can focus on one or two EHRs that you need to integrate with immediately and build those out or work with people that have built them.

Gennaro Maida (31:06.222)
Because it doesn't become cost effective building them all from scratch across the board. It gets harder and harder. So leveraging companies, technology companies that have built them and using the fire, smart on fire, HL7. We're starting to see AI come in because it's able to handle some of the, let's call them, translation problems between one system to another that it's been hard to do algorithmically, whether it's a three-line address, two-line address, the you know, the birthdays being switched, how do you identify that?

duplicates. And a lot of the notoriously difficult problems are something that large language models are able to handle a little bit more natively.

Jesus Moreno (31:48.851)
In working with the advisory body like AHA, where do you see the boundary between accelerating data accessibility and AI accessibility for AI and maintaining strict clinical validation standards? How how do you how do you see those traditional strict

evidence driven bodies adopting AI and and its advantages and disadvantages.

Gennaro Maida (32:31.192)
Sure, you know, and you know, American Heart is specifically working on some of those problems because the ultimate goal is to have personalized medicine. And everyone knows as for years, here's your blood pressure, it's high or low. And that's a bit naive. Some people tend to run a little bit high and it's fine. Some people tend to run a little bit low and it's fine. And you know, American Heart has care plans or care paths that they supply to physicians to be able to supply the patients. And people understand that the it might vary.

differently between patient to patient or based on geographic area or even they're finding out even within the hospital systems are slightly different as well. So you can look at the geographic, socioeconomic, you know, race, age, sex, all of that fun stuff, and start to be able to look at how these things affect that care plan. And then ultimately you really need to look whether it's on a genetic level when we get to the point where we can process that with speed.

and what's going on with a specific person. And what we're finding is a lot of the data right now is for the old internet and older algorithms. So a lot of the work now is translating these huge compendiums of data into something that's accessible by AI, whether it's through MCP or other different connectors, to be able to pull that data and then look at the biometrics that are out there from a patient and being able to fine-tune and move that forward.

You know, and you see a lot of issues with bias, and there's a lot of work being done. You know, I bl one of the white papers I'm working on now is being able to try to quantify bias when you look at the solutions on these large language models or with any of the I guess let's call them AI models, even though they're deep learning computer vision. And a model might be specifically biased for a specific group. That's fine, just be aware that it's specifically biased for that specific group when you use it there.

There was a model that was trained across Mayo Mayo Clinic information and it worked fantastically well in the Mayo Clinic, but when you applied it to another hospital system, it didn't work as well. So you're starting to find out what is the bias and making sure that you pick the right, and which is an additional bias. You know, what are you looking at and testing it for to make sure it's biased? Because no matter where you look, there's always another level of bias, whether it's the architecture of the actual language model, the initial starting point.

Gennaro Maida (34:58.294)
in the weights and then what data you put in and then and it starts to keep going down the road and you start to get getting down to this you know infinite loop but at some point you actually get to a level where you can quantify it. It's just extraordinarily computationally heavy and then trying to decide what are the most important factors you're looking at to figure out where that bias points. Is it race, it is it age is it genetic is it hospital systems? Is it the fact that you're in the Mississippi Valley?

And it might be histoplasmosis you're looking at from the pigeons, you know, one of those board questions from med school. So, what is the most important thing that's creating a lean in that model? And then how do you leverage that lean? You end up with a set of models that are good for space very specific groups, which helps to personalize, and eventually potentially training individual models on a person who knows medicine down the road might be language models, small language models trained on your individual data.

and then update o each day and take a look at your data and update to make sure you're staying on track and have an underlying model that looks at it monthly, yearly, and you can actually start to look at these trends which actually start to be come important. So I think it's pretty interesting and it's opening up and developing. And there's a lot of different groups trying to solve the bias problem, you know, as well as these interoperability problems, how to translate all the data we have into something that's usable to AI, making sure that the data is actually good, you know, and and is actually

useful as well. And trying to find out, you know, from a research side, when you start to see is this something that has been researched before? Is the data actually out there or a paper written? What area is actually the most beneficial for us to start to move forward in? And building engines to actually start to look at that sort of stuff to be able to move things forward at a rapid pace as well.

Jesus Moreno (36:48.146)
And I imagine this is the type of thing you you deal with with your clients at De Novo Innovation Group w when you're trying to prove a clinical claim rather than just showcase a feature, the underlying architecture has to has to change. So when a digital health or meta company comes to de novo needing an AI architecture that can eventually stand up to a regulatory evidence.

a scrutiny from a regulatory body, where do you tell the them to start? Wh w where's this the starting point to building the infrastructure needed for for evidence to speak for itself?

Gennaro Maida (37:27.307)
Okay.

Gennaro Maida (37:35.577)
Depends how big the company is, what their long term goals are. You know, I and just a good example is when you start to look at, let's take Claude, for instance. You can buy an enterprise that membership that's supposedly HIPAA compliant, BAA compliant, but you need a minimum of 20 seats. It's not cost effective for a small company. So the next question is do I build it myself? Do I put that on AWS bedrock and start to build that out? And then the next question from there is all of a sudden, well.

Then I need to build out my own memory. I want that to be secure. And then you have to build out the harness to make sure that you're going in different directions. And then ultimately, one of the things I'm working on now is to take some of the decision or all away from the large language model. It's handle it handles the inference. But when it comes down to which tool I'm going to choose and which piece of data can I access, there's a an actual mechanistic router that makes that decision that the AI can't go around.

we've all see we've all played with Claude and all of sudden it's making a decision. Why did you do that? you're right. I didn't mean to do that. I'm not sure why. You know, so trying to actually control that a little bit better. And I've created a set of tools that I use when I start to work on a new project. I do it I work on a documentation first and I describe the project from an architectural standpoint and the general idea behind the project. And then I have my tools actually pull in from a set of tools that I built.

that are focused on that type of project. And they come in before writing any lines of code or working on infrastructure projects like Microsoft or AWS, that I try to give any AI a complete understanding end to end and the tools to be able to pull the right data off the bat before I even start writing a configuration or a line of code. And then it's focused on that particular company, that particular problem, and then you iterate from there.

And make sure always every time you build something, it checks to make sure it's been achieved, to make sure it's valid, and to make sure it meets the set of rules. I have you in my clot, I have my rule zero is don't make assumptions, always base everything on ground truth and double check everything and validate anything. And that's the first three lines of any and even then I still have to repeat it. Don't make assumptions. you're right, I'm making assumptions. You know, and it has to go back. But I focus very hard on keeping it a tight rein on what data's available.

Gennaro Maida (40:00.121)
To any of the coding partners. And I also have validation. I have Codex as well as Gemini that are used as reviewers, and they actually work as a three-party, let's call it a discussion where they work with each other, and then I get to see what their thoughts are, and then I actually get to make a final choice, getting different inputs from different AIs that have different blind spots. So I cover as many as I can, and based on

You know, MCP pulls from say Railway or Redis or AWS, Microsoft, and it pulls live data so I understand where everything is and make decisions based on that.

Jesus Moreno (40:43.356)
And okay, well, that's that's a huge topic and and it it it really requires a lot of nuance and and a global understanding of the process to connect wire those all those seemingly yeah, all those tools that are very good at at looking like they're telling you what what is true.

Gennaro Maida (40:49.2)
Yeah, thank you.

Jesus Moreno (41:12.714)
But not necessarily so. and wanna engage that that same perspective, but now speaking of how do you design a clinical trial to accommodate for those nuances? How do you work on making a large data compendium accessible for AI? And how do you how does that shape the way you

you use traditional for to for this example, let's let's talk about traditional cardiovascular study data. are you able to to use data that was collected years past by that was collected in a completely different IT environment be relevant to to what AI needs or or expects

in order to derive that clinical

value and move in looking forward. How do you design your your clinical trial protocols to accommodate for AI to to maximize the outcome derived from this technology?

Gennaro Maida (42:34.029)
That's a you know that it's a very interesting problem. It's something I came up I came across when I was working with Cena. And it's something I call, let's call it the epistemic clean room. It's a fancy word for a researcher has access to research data, but also has access to capital and commercial data. A researcher can say, I have a disclosure. I'm not going to take this into account. But when I layer AI on top of it, it doesn't have a disclosure built in.

So you want to make sure that it doesn't make a decision on research based on the fact that you just raised 10 million in capital. So being able to separate those. But it's also interesting if you can give it access to just research and then access to both, being able to say, were my past decisions biased by that capital information now that you have access to it. And you see this on a temporal basis as well. Decisions that were made early on in a trial aren't necessarily applicable to current.

to where you are currently in a trial. But this I think goes more to a data provenance question, which really comes down to the FDA. It depends how that data was collected. From an AI perspective, could it be beneficial internally to researchers to make decisions? Probably. Would it be valid to be able to present to the FDA to be used in a trial? That depends. It depends how it was connected. You know, one of the things that we're seeing, you know, Julio works in Latin America and depending on which country you're at,

Do they have data collection requirements that meet the requirements of the FDA? Most groups go in there and talk to the FDA and say, hey, we're collecting the data anyway. Let's make sure we connect it, collect it to the level that we need to save the trouble of having to redo it when we get to the US. And you know, so you can actually use some of that data and make it reusable. So the question is how it was collected. And for startups, I always say talk to talk to the FDA early. Build that relationship.

They have their doors open, the conversations are free, your pre-submissions are free. And to be able to leverage their knowledge and say, this is how we collected the data, does this work? And whether you know they'll tell you yes or no. And so sometimes it depends what you want to use the data for. And internally being able to guide which way you might go in the future is pro is something that's generally valid for that, but whether or not it's actually usable.

Gennaro Maida (44:58.082)
for trial data is up in the air and that's an open ended question depending on

Jesus Moreno (45:04.584)
Let's say you're advising a founder who's who has a genuinely promising wearable device, but he doesn't yet have a clear platform or plan for validating it against a clinical fact. What are the two or three things they could do or or need to to fix from their data collection pipeline before they take that data?

Gennaro Maida (45:09.24)
Yeah.

Jesus Moreno (45:33.916)
anywhere near a sponsor or a FDA.

Gennaro Maida (45:38.703)
Rule number one, gather everything. You can't go back and gather data you missed. So when you have that opportunity, make sure you're collecting every piece of information you can from that wearable. What might happen is at the time you're collecting it, it might have not been validated. That particular, let's take O2 saturation might be go undergoing the FDA. So you don't know if it's usable, but down the road, they get the 510K, they get the approval or authorization, and then it clears.

But if you didn't collect it, all of a sudden you missed out on that whole piece of data. And it doesn't mean that you can't use that to be able to influence your choices. And it's becoming more commonplace. And it's a you know, it's a gray area, but there's a lot of wearables out there. Say Fitbit. They last I checked, they had a website up and they had over 700 studies based on Fitbit, but at the time it wasn't clinic it wasn't FDA approved. So the question is there clinically validated.

versus FDA approved. They're different things. And a lot of the wearables that are out there and the data that's being the data that's being used and gathered, it has been clinically validated. There's been a ton of papers to show that it's pretty accurate. And is that use usable? That ends up being again an FDA question. Are they going to let that slide through? Has that device been used before? Are you building a completely novel device? In which case you want to have a device that's validated while you're wearing

your device to be able to validate it. Make sure that you're getting some sort of ground truth data. If you're looking at heart rate variability and peak detection on a PPG, make sure you're getting an EKG if you can, because the peaks are sharp. It's very good to be able to pull those peak detections. Using a clinically validated oxygen saturation, you know, blood pressure cuff that and you know whether or not you actually need to do you know arterial pressure to be able to validate blood pressure for a patch. There's different levels.

But find some way to get ground truth from a well-known device that's out there or a heavily validated device that you can base yours off of. Otherwise your data is just sitting there with nothing to attach it to. And gather all the data.

Jesus Moreno (47:45.428)
Gold standard.

Jesus Moreno (47:49.716)
have the gold standard on hand to to compare it to and and point towards you know closeness in in results.

Gennaro Maida (47:59.801)
Yes.

Jesus Moreno (48:02.13)
And looking ahead, where do you see the ground truth problem in digital biomarkers actually getting solved? Is it gonna be better sensors, better AI validation standards, or maybe even a regulatory shift?

Gennaro Maida (48:17.238)
Mm-hmm. That's a good question. Part of it is, I think, regulatory shift, which we're seeing with the predetermined change control plan that allows you to say, hey, here's going to be our next 10 changes with AI. I think there's going to be improvement on the small language models that will be focused because ultimately when you look at a biometric, it looks like the stock market. It looks, it's a piece of data. It's a language in and of itself. So you can train it.

And if you look at the body, you have all these different lines. It's almost as if there's a symphony. And can you get something that's smart enough to be able smart enough, robust enough, being able to process fast enough to make all those connections? You're starting to see it with digital twins where you build the structure of the heart and then you're layering the electrophysiology, and you're layering on top of that the the cellular effects of the biochemistry. For instance, if there's a calcium channel blocker, it's going to change the electrophysiology.

Which will change the structure and building these layers upon layers. So part of it's gathering data and finding ways to attach it and then being able to s look at connections in between, in the case of biometrics, in between the different biometrics, because ultimately AI, its pattern recognition, is very, very good at. And making sure you're finding the right patterns, being able to discard that those that aren't. And what additional piece are we missing? Is it genetics?

Then you start to get to the epigenetics, which gets more difficult. So I think down the road it's going to be ga data gathering on a personal level and being able to get enough data. Actually, it's not even more data, it's the right data and trying to figure out what it is. You might have, you know, p petabytes of data and it turns out there's only two biometrics and maybe one genetic, it turns out to be connected. So learning what the right data is out of that gigantic pile of data is

Probably the most difficult.

Jesus Moreno (52:57.11)
Hello, sorry about that. Can you hear me? I'm so sorry. It it it my internet went down for some reason. it happens. Speaking of which, yeah. Mm-hmm. sorry, I I I was so engaged in in your answer I I lost track of of it with with this this whole thing. so I'm I'm trying to find my bearings.

Gennaro Maida (52:58.402)
Left me. I can't.

Gennaro Maida (53:04.844)
It happens. It it it it technology. So

Gennaro Maida (53:22.944)
You know, and I was I I could I I had just finished a really good point. Honey, what was the point?

Jesus Moreno (53:27.434)
Yes, yes, you you were. I was excited. I was walking walking with you, going to that build up and then I was he's gonna, you know, solve the problem in the next couple of of sentences.

Gennaro Maida (53:40.047)
Solve all problems. You know, because it it's funny because you I mean it you're actually I I put together a bunch of stuff that I'm actually I've been working on. It's actually the the evidence governed AI systems. It's the question we're talking about, and a multiple different projects that I have that I'm trying to pull the evidence together to really focus on how to get the ground truth, whether it's that epistemic clean room, being able to guide, you know, what the AI can see and

You know, being able to understand that you're actually pulling valid data and with the validation and the provenance and being able to tell you where you where you found it, sort of like you see with the threat assessments or the risk assessments, and being able to point directly not only to the HIPAA rule, but also we set it here in Microsoft, we set it here. This one is unvalidated or it's left open currently.

Gennaro Maida (54:51.63)
Still here.

I mean, make sure that's not my internet. Since I I did just repo.

Jesus Moreno (54:56.873)
I'm I'm so sorry. I don't know what what's going on. and it's that was actually my last question. So I hope the platform was able to capture the vr your answer. I'll work on the outro of the video on my end. so if you could maybe just give us a a a short you know, thank you and goodbye.

So so the video the editing video team can can work that in and you know clean up all this mess.

Gennaro Maida (55:36.446)
absolutely. No, no. And and thank you, Jesus, for the invitation and by extension, Julio as well. I really appreciate the opportunity to discuss what we're starting to see, a lot of the changes in AI and where they're going.

Gennaro Maida (55:55.372)
You really should do the last two questions again. 'Cause he's not I think it's still recording.

Gennaro Maida (56:05.678)
Good, I can do it again. I'll do it again because I didn't really like it. So no, I want to thank you, Jesus, for your time as well and inviting me along and by extension Julio as well. It's been it's been a lot of fun having an opportunity to discuss a lot of the changes we're seeing in AI and the difficulties applying them in studies and biometrics and making sure they're really grounded in reality. Again, making sure that we're not hallucinating and making things up.

Jesus Moreno (56:07.03)
What's happening?

Okay.

Jesus Moreno (56:33.006)
Thank you, Enado, for for sharing with us all this you know, insightful and thoughtful and technically complex yet intriguing aspects of of technology and how it integrates into the healthcare system. I would love on a separate note, this is me personally, I would love to connect with you. I'm a biomedical engineer. Recently transitioned to the States, trying to find my bearings.

I'm in located in California, so if if you wouldn't mind, could I reach out to you and and ask for some guidance as to how to, you know, work my way into the field?

Gennaro Maida (57:13.324)
No, absolutely. Not a problem whatsoever.

Jesus Moreno (57:15.415)
I appreciate that, sir. I again thank you for your time. Thank you for participating. This was such an intriguing and and engaging conversation. I

Gennaro Maida (57:35.296)
no, that's right, right. Yeah, no, I'm poor guy. It's not us, is it? I don't know, I did just rebuild.

Gennaro Maida (57:58.028)
yeah, yeah, yeah. No no no worries whatsoever. No, it's been fun. and ultimately the the podcast is only thirty minutes long, right? so yeah, plenty of material out of that to be able to pull thirty. Yeah.

Jesus Moreno (58:04.816)
Yeah, w it I mean it I I believe so. Yeah. I believe the team has what they need to to come up with a excellent excellent episode. I'll let you know or or the team will let you know when that's ready. If you if you want to see it before it gets published, please let me know maybe through an email and I'll make sure to tell the team.

Gennaro Maida (58:27.48)
Sure. I decide whether or not I want to put it out there or not. Dude, that was horrible.

Jesus Moreno (58:30.896)
I I would be very sad if if you decided not to. Because it's I think it was a great conversation. I was engaged right until this started happening. but anyway, Jonato it's been a pleasure. Thank you for your time and I hope to yeah, stay connected.

Gennaro Maida (58:42.36)
Yeah.

Jesus Moreno (58:53.008)
Appreciate it. Thank you so much. Have a great rest of your day. Bye.



Jesus Moreno (00:01.112)
Henado, the through line today is one I believe every trial sponsor, MedTech founder, needs to hear. A wearable signal isn't clinically evidenced until someone has done the hard work of proving it. Skipping the ground truth validation and data engineering groundwork does not just invite regulatory rejection. It burns precious trial capital data and

Time. Thank you for walking us through what works in real life and actually brings about innovation and moving the boundary of technology further.

Jesus Moreno (00:47.659)
And to our audience, thank you for tuning into Global Trials Accelerators. If this episode challenges how you view your trial strategy, share it with a colleague or a clinical lead who needs to hear it.

Jesus Moreno (01:25.062)
Thank you for tuning in to Global Trials Accelerators. If this episode challenged you or how thank you for tuning in to Global Trials Accelerators. If this episode challenged how you view your trial strategy, share it with a colleague or a clinical lead who needs to hear it. Subscribe where you listen to the podcast and we'll see you on our next episode. Until next time, keep accelerating.