By Robert Rosenbloom, CEO, Medlive and Jason Olivieri, VP of Outcomes, Medlive
Supporters fund CME to facilitate the adoption of evidence-based medicine. While providers, in turn, develop CME utilizing evidence-based educational strategies for HCPs, its effectiveness in reaching the right audience, with the right information, at the right time is not guaranteed.
Accordingly, educational outcomes data is a critical component in understanding both what works in CME and how it needs to evolve. Traditionally, reporting of outcomes data lags many months behind a CME activity release, which can create challenges in strategic decision-making about future CME support.
Jason Olivieri, Medlive’s VP of Outcomes, has watched that lag play out for years. Robert Rosenbloom, Medlive’s CEO, saw the need for a new standard in the industry and made RTI a priority: the Real-Time Insights (RTI) Dashboard, a platform that surfaces engagement data in real time.
We sat down with Jason and Robert to hear how RTI came together and what it changes for supporters.
Q: Take us back, what was it like for supporters before RTI existed?
Jason: A typical scenario for digital, enduring CME was to deliver a PowerPoint-based outcomes report approximately three months after activity launch. This first impression likely only included two months of information (because it takes time to pull and clean data, conduct analyses, and build a PowerPoint report). So, overall, we had a labor-intensive process to generate insights on not a lot of data. It wasn’t ideal. By the next report (3 to 6 months later), there’d be enough data to have a more robust understanding of CME effectiveness, but the supporter may have moved on to the next planning cycle – thereby potentially missing key insights.
Q: What does it look like now when a supporter gets asked how a program is doing mid-flight?
Jason: When a supporter wants to know how an activity is performing, RTI delivers up-to-the-minute data: educational reach, learner demographics, pre/post assessments, engagement metrics, and learner perspectives on their current practice. All of it (quantitative and qualitative) is accessible via a single web-based dashboard, on demand. It’s a real shift in how supporters can relate to the activities they fund. In the past, supporters received a summary of what providers determined was most relevant to share. With RTI, they are looking at the same data we are. That is a fundamentally more transparent relationship between Medlive and the organizations that support our programs, and we think that is how it should work.
The RTI Dashboard gives supporters an at-a-glance view of audience reach, learning gain, patient volume, and the full CME learner funnel, updated continuously as the program runs, with 170 days still remaining in this activity.
Q: What made engagement validation from verified HCPs the thing you had to solve before anything else?
Robert: Engagement validation is one of the most important things supporters should be evaluating on any reporting platform, yet there’s often no consistent industry standard for how that engagement is validated. RTI includes an engagement validation algorithm that filters out bot traffic, security crawlers, and other invalid traffic sources before any data surfaces to the dashboard. What you see represents verified HCP activity. When those numbers go into compliance documentation, grant justification, or upstream reporting, they need to be accurate, so we built in that transparency and validation layer from day one.
Jason: From an outcomes standpoint, this is where RTI provides real credibility. Everyone, not just supporters, needs to work from real HCP learner data. That our engagement data has been validated is not just a differentiator. It should be a standard.
Q: How granular can supporters get when they need to understand impact on a specific audience?
Jason: RTI gives supporters an in-depth view across multiple data layers. On the engagement side, the dashboard surfaces the digital footprint of each learner: time spent in education, the specific points where learners are most focused or drop off, and polling response data. All of this can be filtered by learner demographics, so supporters can see how specific cohorts moved through the content and where their attention was concentrated. For more traditional outcomes data, RTI includes full demographic breakdowns and location mapping at the state level. Pre/post assessment results can be filtered by degree or specialty, giving supporters a clear read on knowledge shift within the populations that matter most to them. On the qualitative side, open-ended learner responses are categorized both thematically and by frequency of mention, so supporters can see which themes surfaced most consistently across the learners. That layer of qualitative insight is often where the most actionable signals live. We have also designed RTI to iterate. There will be additional reporting and filtering capabilities coming in the months ahead.
Robert: What I want supporters to understand is that the filtering and segmentation in RTI are not simplified views. Every filter reflects their exact population, backed by full statistical analysis. That matters because when a supporter needs to report on outcomes for a specific specialty or geography, the data has to be accurate enough to bring to a compliance team or medical affairs leadership. We held that bar high when we built this, and it shows in how supporters are using it. The behavioral and qualitative layers Jason described are surfaced in the same filtered view, giving supporters one coherent picture of their audience rather than requiring them to piece it together across separate reports.
Learning gains are presented for both intended and other learners. In the photo, clicking into the Intended Specialty block opens a detailed cohort view, where one can filter and organize learners by attributes like degree or profession to see exactly which groups are driving knowledge gain, with the option to download the full dataset as an Excel report.
Q: How did the AI-powered insights end up in RTI?
Robert: We did not want to just hand supporters a pile of numbers and call it insight. Getting here took years of work: we first had to build the data and ETL infrastructure to consolidate and clean learner data before we could layer AI on top of it to help uncover meaningful insights. Without that foundation, the AI piece simply wouldn’t be applicable. So, every quarter, RTI generates an AI-powered analysis of content performance across the program, identifying strengths, knowledge gaps, and anomalies in the learner data. The AI output is reviewed by Medlive’s clinical experts for accuracy and relevance before it goes to the dashboard. That review step was non-negotiable for us. The goal was always to give supporters something they could bring to their leadership and act on with confidence.
Jason: From the outcomes side, the case has always been clear. We were spending a lot of time cleaning, analyzing, and packaging, which a well-built system could do faster. What took longer to sort out was a reliable output. Furthermore, where AI is leveraged, we needed a process to ensure the results were trustworthy. And that’s why clinical expert reviews are still involved. Despite the fully digital packaging, every AI insight is reviewed by Medlive’s clinical experts for accuracy and clinical relevance before publishing to the dashboard. In other words, AI surfaces signals, which our clinical team vets, so that the resulting RTI dashboard is something supporters can confidently trust.
Each quarter, the RTI Dashboard’s AI Insights layer surfaces AI-generated insights vetted by Medlive’s in-house clinical experts that translate program data into specific, evidence-based recommendations for optimizing future content strategy and learner outcomes. In the photo above, open-ended learner responses are converted into thematic categories, surfacing the topics and clinical questions most frequently raised by HCPs, with a vetted key takeaway and representative quotes, so supporters see the full picture without manually reviewing individual entries.
Q: Looking ahead, what can Medlive’s clients look forward to with RTI?
Robert: Right now, we’re at single-activity dashboards, and that’s already changed how supporters can act on program data. But that’s just the starting point. What we’re building toward is longitudinal reporting across a full curriculum or initiative, so clients can see how learning builds over time, not just within one activity. Beyond that, we’re building aggregate views spanning multiple programs and therapeutic areas, so a supporter funding several initiatives can see the full picture in one place. The foundation we built from day one was designed to scale into that as programs grow, so that the transparency and convenience of the RTI Dashboard become standard across every Medlive program.
