Data science is amazingly powerful, and yet there seems to be a brick wall that data science teams routinely hit: the model deployment and monitoring phase. This morning, Pecan released powerful new capabilities that not only make those critical steps in the process much easier to implement, but also supercharge the role of the data analyst even more. While data analysts could always use their SQL skills to build sophisticated ML models in Pecan, now they can also readily manage those “last mile” tasks of model deployment and monitoring, all within the Pecan platform.
We talked with Pecan’s CTO and co-founder, Noam Brezis, about what these new features do and how they’re critical to fully granting data analysts the power of data science.
There’s a variety of stats out there about how few models actually get deployed after data scientists pour time and energy into them — and they’re all pretty discouraging. Why is model deployment so often such a major obstacle in the data science process?
Indeed, the commonly quoted data point is that some 80% of data science models never get to see the light of day, meaning that they don’t get to be used to support a business’s success. These stats are really depressing for those who put a lot of effort into manual data prep and model building. Data scientists can put months of work into a model that never actually gets used by your organization. This is partly because a lot of companies have hired data scientists but haven’t also built a team of MLOps engineers who are ready and able to support them in integrating those models into all the surrounding systems that need to work together. And data scientists, while very skilled in statistics and machine learning, don’t typically have the full range of engineering skills needed to bring their live data sources and business systems together with their fully built models. Production data also ends up working differently with models than data scientists anticipated with test data, so results can be poor.
Overall, it’s a frustrating situation for everyone involved. So that’s why we knew it would be crucial for Pecan to have a very simple method for deploying models — and we’re excited to say we’ve got it down to one click, streamlining that process incredibly.
“We knew it would be crucial for Pecan to have a very simple method for deploying models — and we’re excited to say we’ve got it down to one click.”
We shared that Pecan is also now able to monitor models in production. Why does model performance sometimes change over time? How would a data or BI analyst use this functionality to make sure models are still doing their best?
Everything in our world is constantly changing, right? We certainly see that happening these days. And the same thing is true of data. When a predictive model is built and evaluated on one set of data at one point in time, it’s mathematically optimized to make predictions based on how things were in the world at that point in time, as well. But when the model is in production, the data coming into the model starts to change – because the world is changing – so the model may need to be adjusted as a result of the data drift. Sometimes the outcome that you’re trying to predict is also changing because your business goals have shifted. That is also a great reason to monitor and revisit your models’ performance – because let’s not forget, our goal is not just to create accurate models on test data, but rather to generate predictions that enable the business to achieve a real-life, specific set of KPIs.
Successful data science requires ongoing monitoring and adjustment of models in production to ensure they’re still providing high-quality, accurate predictions. Again, this is something that MLOps specialists might do at an organization with a large data team. However, Pecan is bringing that ability to the BI or data analyst who’s closest to the data, closest to the business goals, and allowing them the capability to monitor and adjust as needed. Pecan gives analysts capabilities that used to be reserved only for specialists and makes sure they can keep their team’s needs front and center.
“Our goal is not just to create accurate models on test data, but rather to generate predictions that enable the business to achieve a real-life, specific set of KPIs.”
Another important step in achieving business impact from data science is simply getting the output of a model where it needs to go. So what’s the best way to make sure the people who need the predictions can use them in their day-to-day programs?
Even if you have a fantastic model that performs really well and addresses business needs, it’s just useless if it doesn’t integrate with the other systems a business uses day to day. For example, marketers may need a model’s output — the predictions — fed into a CRM to direct an upsell or a winback campaign. For an app publisher, you might need predictions sent to an MMP to improve attribution measurement or to shape targeting for a Day 7 retention program.
Wherever it makes the most sense for your team — we want to get actionable predictions there for you. And like the other steps of the process we described, this one also needs to be easy and straightforward. We’ve built integrations for many standard business systems and continue to expand the list of options based on our customers’ needs. That means that the people who are best positioned to take action on the predictive scores generated by Pecan will have immediate and convenient access to the latest information, so they can make the best decisions possible.
What are you most excited about in this round of updates to the Pecan platform, and what are you looking forward to working on next?
Personally, I’m extremely excited about the ongoing live monitoring capability. It’s a paradigm shift that allows us to be confident that the models continue to be highly accurate over time.
We’re also working on ways to make the model-building process even more accessible to analysts with SQL skills through a refined and friendly interface. Stay tuned for more details on that soon!
Ready to supercharge your team’s data analysts with full data science powers, including these new model deployment and monitoring capabilities? Pecan can advise on how to bring AI into your team and company for maximum business impact.
Get in touch for a quick, easy use-case consultation. We’ll help you find the best way to get future-ready.
