How to Assess AI Maturity and Build Impactful AI Solutions

Samuel
Hey everyone, welcome to Mastering AI with the Experts. Today we're diving into an exciting topic: how organizations can get AI ready and really make the most out of their data. Joining me is Neha Sharma, a datand AI specialist at Microsoft. She's all about helping public sector and large organizations modernize their data platforms and governance and unlock some seriously impactful AI solutions. Neha's got a ton of experience with tools like Microsoft Fabric and app modernization, but what makes her stand out is how she connects the dots between cutting-edge tech and real-world results. Plus, she's super curious about philosophy, which gives her a really thoughtful perspective on all of this. In this chat, we talk about how to figure out your organization's AI maturity, why clean and well-governed data is a must, and some awesome examples like what Kelowna is doing with AI. It's a practical, no-nonsense conversation about making AI work in the real world. Let's jump in. So first, thank you, Neha, for taking the time to meet with me today to talk about AI maturity, to talk about data. I think it will be a super interesting conversation. A lot of customers are asking me, how do I know how mature I am? What kind of AI tools should I use? Is my data ready to handle all those complex AI models? So it will be a super interesting conversation. So thanks a lot for taking the time to meet today.
Neha
Yeah, super excited, honestly. And I think this probably one of the most critical topics, right, when it comes to AI readiness. So excited to tackle it with you.
Samuel
Awesome. So let's jump right in and start by talking about AI maturity. So how does an organization know where it stands in terms of maturity, not just limited to data, but overall? So how would you define an organization's maturity globally to implement AI?
Neha
Yeah, that's a super great question. I think AI maturity really comes down to an organization's ability to leverage AI solutions and tools and integrate it into their operations. So you're doing it maybe today, but then are you getting a high return on investment?. What is the value that real people in your organization experience, right? So being able to have increased productivity, more time focused on meaningful work, that's how you want to measure it across the organization. And it's not only inclusive of the technical capabilities, and that's what a lot of folks focus on, but it's also the data readiness, the organizational culture, for example, that you need to have behind it, strategic alignment across the business, right? So that to me is what's representative of a holistic, entERPrise-wide AI maturity.
Samuel
Can you walk us through, I don't know if there's a model out there, but can you walk us through the different stages of determining the level of maturity of an organization?
Neha
Yeah. At Microsoft, we actually have a framework that was released a few years back. And so essentially there's certain levels of maturity. There's about four tiers that I would kind of put organizations within. The first is foundational. So these are customers that recognize that AI is here and they recognize AI's potential. Maybe they're a little bit skeptical of using it, right? So they're still in that evaluation stage. They might not be confident about what it's actually useful for, so they need to still do a little bit of discovery within and figure out what those use cases are. Then there's the second set of customers, so they're approaching, that's what we kind of call them, and they're curious, they're experimenting. They're in this era, sort of like City of Kelowna, right? So you saw some studies kind of go out the past few years. They know that they might get their hands dirty. They want to look at use cases, they want to assess its value, and they might not realize it right away, but they're in that experimentation stage. And then there's aspirational organizations, right? So ones that have some custom-built AI solutions, they have implemented them, but they're not in every area of the business. It's not critical core business processes. It's only areas where maybe it was easy to realize value from. They have a culture that's supportive of AI. They want to advance that as part of their business strategy, but they're not necessarily mature yet. And then of course you have some of the more mature companies, and that's that last sort of tier in that phase. And these are companies that have custom, advanced implementations of AI, right? So they're leveraging things like, I've got videos of my infrastructure. If, for example, Canada's infrastructure, let's take the government, right? If there's any leakage in terms of pipes across the infrastructure, if there's any cracks in some of the sewage pipelines, can we identify that? Can we then take it back and be able to respond to it quicker, right? Because there's a certain age of that infrastructure that's coming to an end. So these are again specialized implementations, and this across the fundamental mission of that organization. This changing the way they operate, the way that they process things, the services they offer, and it's built into that core process. So those are the really sort of mature customers.
Samuel
From your observation, what percentage would you consider being in stage four, or even in every of those stages?
Neha
That's a really good question. I had looked at some Gartner studies and I spoke to a friend actually at Gartner, and I would say less than 10% definitely in the stage four. But 10% of organizations are maybe even at a stage that's level two or higher. So a lot of the companies that we're dealing with today, 90, 95%, are approaching in that stage. They're experimenting, they're still trying to figure it out, they're looking at what they need to establish internally to advance to some of the next stages. But I would say it's very much aspirational I think is the next goal, maturity-wise. Very less, less than 1%.
Samuel
Right. So that's interesting because while talking with customers, I realize that a lot of people have this fear of missing out, like the feeling that every organization is super mature, but the truth is most of them are not. It's still fairly new, and adoption rate has just increased at lightning speed in the last two, three years, actually since ChatGPT model 3.5 Released. The discussion around AI has just increased monumentally, so it gives this impression to entERPrises that every other entERPrise is at the top of their game implementing those tools, right?
Neha
Yeah, absolutely.
Samuel
You mentioned earlier City of Kelownas a use case. I do know the use cases, but for people listening to us, can you maybe expand on what they did achieve? I think you mentioned they were on stage two, right, experimenting with AI. So can you expand a bit on the Kelowna story?
Neha
Yeah. So Kelowna I think is tackling AI through multiple different initiatives. Some of this of course building a custom solution, but some of this also looking at what's already available from a Microsoft perspective. So things like Copilot, right? Turning that on or trying to figure out a way to implement that across the City of Kelownand the users and advancing that productivity there. The second piece to this when it comes to custom solutions is Kelowna experimented with building permitting solutions. So essentially there's a building permitting process that takes place, and in that process we wanted to make it easier for companies that are performing constructions across the city to be able to submit those forms, evaluate that, and speed up the process to actually getting that approval in order to be able to implement and build buildings and infrastructure across the city.
Samuel
Interesting. Yeah, and I did work with Kelowna, so I know how innovative they are. A lot of people are thinking that public sector is not innovative, but I can tell you in Canada it's not true. And Kelowna is a real proof that.
Neha
Yeah. And actually, Government of Alberta, if I may add, this super new, but they just released, I think two days ago, it's live, they released an AI search. So Alberta now, if you log on, they actually have like an AI search, alberta. Com or something. But essentially you go in and anything that previously was available on their website, any policies, procedure, grant programs, permitting, any questions you have for that Government of Alberta, you can just go in, log into the portal, you have an interface where you can actually go and ask some questions. So I've played around with things like, hey, I want to look at grant programs, for example, because I'm building a new organization or a new company, I have a startup. And it actually lists out all of the different grant programs that are available, the size, the scope, and here's the references. So you can actually go back to those programs and figure out, okay, which one's actually a good fit. But that makes it super easy, right? Because you're not actually just going to the Government of Alberta website anymore and trying to figure it out yourself. You're actually just engaging in natural language with the knowledge that already exists on that website.
Samuel
So I will just assume it will remove so much pressure on their customer service agents. I mean, I don't know about you, but personally when I end up on my city website, it's so complicated, there's so many subsections that most of the time I just give up and I just call in. So I will assume that this use case specifically will bring tremendous value for the citizen, but as well for the government or the cities that are implementing those solutions.
Neha
Yeah, absolutely, right. It's about how are we servicing Canadians. That's the question that they're trying to answer, and this makes it a little bit better to engage and be there in the time that Canadians are reaching out and need them.
Samuel
So you described this framework of AI maturity, but now how might an organization be able to identify basically where they stand in this framework?
Neha
Yeah. So I think there are key pillars that you want to use to assess your organization's maturity, right? So any customer, I would have them look at things like AI governance, data management, and then I always go back to this framework, which is people, process, technology, right? So the questions here that you might ask are directly tied to something I actually often refer to, and I actually want to share my screen and show you super quick, but it's called the Lippitt model. So this a framework for managing complex change, and it was actually something that Dr. Mary Lippitt in 1987 released, but essentially helps answer some of these questions. And it's applicable across various industries. So let me share my screen. Let me walk through it. I think there's five components, and by the way it's applicable across a lot of different industry, different change, different situations. But in this particular case, the questions I would look at and the way I would approach this these are the five components that you need to achieve success when it comes to change within an organization, right? So if any one of these components are missing, you'll often see you experience confusion, or perhaps your employees are anxious, or there's some gradual change that needs to take place, there's some frustration or a false start, and it depends on which piece of the puzzle you're missing. So it helps you really evaluate that. But the questions here are, do we have a clear vision or long-term strategy for AI and how AI can add value to our organization, right? The second here is, do we have the skill set to support building AI applications and the necessary datand technology infrastructure as well that goes with those AI applications? Do we have incentives in place, for example, to drive these projects forward? Does our C-level executives, do they care about this, right? Do our investors care about this? What about the right processes and resources? Do we have that in place? Do we have the required technology infrastructure supporting it and supporting digital transformation? And that's going to get you really close to that action plan.
Samuel
So for people listening in and that are not looking at the video right now, basically we have this matrix where we have those five components of change: vision, skills, incentives, resources, and action plan, as you just mentioned. And from what I'm seeing, if I have the five of them, I'm on the road to change, but if I'm missing, for instance, the vision, I can create confusion, and if I'm missing the skills, I can create anxiety. Am I getting it right?
Neha
Exactly. Exactly it. So whatever it is you're missing, there's going to be a resulting response to that. So to change, you need all of those five components. You need a vision, you need skills, you need incentives, you need the right resources and processes in place, and you need to have a clearly defined action plan. Any of these things missing, you're going to come across some sort of mismatch and you're not actually going to have successful change. You might have something that's a little bit off from where you want it to end up.
Samuel
Okay. That's a really good framework. So for people listening in, we will drop the matrix in the show notes. Now, if we continue, what are you seeing as being the common challenges for organizations when they're trying to reach this AI maturity, and how can they overcome it?
Neha
Yeah. I recently saw, Sam, a video of Tim Ward, I think, and it was the CEO, co-founder of CluedIn. CluedIn is an MDM platform for anyone that isn't familiar, but I think he captured it really well when he was having this discussion. And he said he met with a CDO of one of our joint customers. So they work closely with Microsoft, and he said that the customer essentially went to their C-level executives who were kind of enforcing and saying, hey, we need to have a custom Copilot. And that's all good and well, right? But the CDO cares about having the right data quality and making sure that there is a lot of value that we can get out of this initiative, right? Not just building for the sake of building. So to get this message across, she essentially said, I can build you a custom Copilot that answers with answers that look like this, or answers that look like this. And the first being where data is being pulled exactly as it is, right? There's nothing that's been done here. It's their current data landscape. This here was data quality rules are in place, sensitivity labels have been applied, there's a single source of truth with the MDM that they have as a solution. It was clear that this sort of, you just graduated your AI solution from a high schooler to perhaps maybe a university graduate, right, to put into perspective the quality of the responses, the context, and the way that it's actually going to help add value for the organization is starkly different depending on the amount of work that goes into building that data governance framework and investing the time there. So I think that's where I find that organizations really struggle. And to reach a level of maturity, you need to have that be understood, that it has to start with data governance. You need to know that if you're investing money in AI, that also goes along with investing in a data strategy, and it also means that you need to implement a master data management solution.
Samuel
Yeah, that's something I see a lot with Microsoft 365 Copilot as well, where people are asking questions but don't get the answer they want because the AI basically is pulling too many data. It's just pulling too many irrelevant documents, and at the end of the day, garbage in, garbage out. If you get the wrong document because you have too many, you'll get a bad answer, right? I mean, I think it's important to understand that LLLMs or tools like Copilot are basically using context to feed the LLM, so it understands how to provide you the answer you're looking for. So if we take as an example plugging Copilot on SharePoint, but then you drop like 12,000 documents, you didn't clean them, you don't even know what's in it, probably the answer that you'll get from the Copilot won't be exactly what you expected. So I do agree that data cleanness is super important, data tagging as well, having a general data governance, which basically brings us to our next subject, which is the data infrastructure. So what exact role is data playing in an organization using AI solutions?
Neha
Yeah. I think in terms of the data infrastructure, we need to think about the foundational tools and systems essentially, right, that enable the collection, storage, management, and processing of the data. So what you want to address with a robust infrastructure is common challenges or problems like data silos that exist currently in your datarchitecture or data platform, right? Data silos, the issue with that is it creates integration complexity. So you want to eliminate that as much as you can. Also eliminate data quality issues. You want to address a lack of a golden record or a single source of truth. You also want to take a look at things like data latency, right? So the increasing costs as well that come with scaling those datand analytics capabilities over time across the organization. And then of course protecting the sensitive data, the PII data, that's absolutely crucial. So you need a platform that's going to help alleviate all of these different issues, and it's not going to be a single solution. And this where the datarchitecture is all of these key components bringing it together to be able to achieve and solve for these issues and problems.
Samuel
Yeah, totally. And again, you mentioned security. It's super important. You can't just drop a bunch of data without validating, because that's something I'm seeing a lot. If I come back to my example of dropping 10,000 documents in SharePoint, what happens often is that we didn't even validate if people should have access to this data, right? And then you end up with Copilot providing information unclassified or information that you don't want. So it's super important.
Neha
Yeah. And like which users have access, right?
Samuel
Exactly. As you said, what are the key components to build this data infrastructure, right? I will assume it's just not one product. When we understand that most organizations have several systems from where they want to pull data in, how do you manage, in terms of infrastructure, this data governance?. So I will assume there's multiple pieces of the puzzle, right?
Neha
Yeah. So let's say it starts with a robust data strategy, right? And this where we define the solutions we need, because you need to get clear on exactly which pieces of the data strategy that solution is going to help tackle. So think about the first one as data security and compliance. In this case, you want to have policies, measures that you want to put into place to avoid unauthorized access, you want to prevent breaches, you want to avoid misuse of some of the data, and you're also adhering to the relevant laws and regulations for your industry. So in my case, I work with healthcare customers, so they care about HIPAA compliance, right? There's GDPR compliance. So all of that and those regulations, that's what your data security and compliance tools are going to help you to map towards. And then there's encryption tools that's also part of it. You want to have access controls, you want to have classification, safeguarding sensitive information. So that is all under that data security and compliance umbrella. And then there's master data management. So oftentimes these terms, they get a little bit confused, but I think this the umbrella that I like to think of it as. So master data management is a little bit different. It includes a process or tool set that's going to help you create a single, consistent, accurate source of key data in your organization, right? So this often referred to as the golden record. You're integrating all of your data from your various sources, and what you're doing with a tool like CluedIn or Profisee is you're actually helping to resolve those inconsistencies. You're actually going to help eliminate duplication, right, to be able to have that single source of truth.
Samuel
And I know I gave examples like documents, like PDF, in my previous example, but the truth is this master data management and the infrastructure we're describing, it's data coming from multiple systems, right? So it might be documents, but actually, can you give an example of what kind of data might be useful for an AI initiative? Do I pull information from my ERP, from my HR system?
Neha
Yeah. So I mean, it really depends on the use case, right?. But I think when it comes to the governance piece and the master data management and security compliance, it has to span across everything. So ideally what you want to do is think about all your different data sources, land it into a single storage format or a single storage layer. And this super critical because a lot of times when you're dealing with a traditional datarchitecture, what you have is multiple different sources, a lot of different subsystems, and all of these have different ways to secure them is kind of what I mean. You don't have one security model. You want one security model to span the entire data estate. And that's where I think when you think about things like MDM and you're implementing it, you actually want everything together in one storage place, one format. And what you want to do is you want that security model or that MDM to secure everything, every single source. That's what you want to implement because that's the way to achieve that golden record, right? Because you're integrating from various sources. So that could be your ERP systems, it could be your unstructured data, PDF documents, videos that you collect, for example for parking space infrastructure, sewage pipes, anything like that. And then it could also be things like perhaps you have social media, right? You have some of the data that's coming from social media, you want to conduct sentiment analysis. Again, before it gets to that, you need master data management. You want a single source of truth. You want to protect all of your records and make sure that's in place. There's no inconsistencies before you get to something like implementing an AI solution for that use case.
Samuel
That's very interesting. I never thought of video or even social media reaction. That's a really good point. So it's really data coming from all over the place, but that can bring value and that can feed your AI. Now we're both working at Microsoft, so from the Microsoft standpoint, what's the MDM tool you're recommending normally to our customers?
Neha
Yeah. So MDM is one that we actually don't cover under Purview, right? Purview is our sort of data governance solution, and it includes the risk and compliance piece, it includes data security, it includes things like data catalog, data lineage. Those are the pieces that we're really good at, and we just released data quality as part of it, so you can kind of set those data quality rules. MDM is one that we integrate with. So for master data management tool, you actually want to look at things like Profisee. That's one that's been around for a number of years. And then also CluedIn. So those are ones that integrate really well with Purview as well as our data platform as well, which is Microsoft Fabric. So that's what you want to look at.
Samuel
From the point of view of a company who just starts exploring MDM, how much of an effort might it be? I will assume that if you're having, I mean, I've seen organizations with a hundred systems, right? What's the effort it represents, and how can you prioritize which one you're integrating first? I will assume you don't do, if I go back to my example of a hundred systems, you won't do all of them at once, right?
Neha
Yeah. So normally what I'm seeing is even if an organization wants to go from on-premises sources and they want to migrate to Microsoft Fabric, let's say as a platform, you want to actually implement MDM on your on-prem. That's totally fine. That could be phase one. And then you just start with whatever is in your entERPrise data warehouse, right? Start to tackle those sources. Essentially there's a roadmap, a plan you can put into place. You talk to some of the guys here at Profisee or CluedIn, they can kind of help build that roadmap out. And then you work closely with a partner maybe about a plan that would be in place to essentially go through everything, look at what's in your entERPrise data warehouse, and start establishing a set of rules, creating that golden record. And then from there, any rules or any of those things that you've kind of set in that MDM solution, you import it over to, it's not as easy as I'm making it sound, but you can take exactly what you build for your on-prem and you can just bring it over to Microsoft Fabric, for example, when you're ready to migrate from on-prem to Microsoft Fabric.
Samuel
So is it the platform that's there more for unifying data? Let's say I'm having a CRM and an ERP and an e-commerce platform, and my customer has one record in each of those platforms. Is it a way of unifying them and making sure that we do agree on the data? So if my phone number is different in the three platforms, I want to make sure it's the right one, or it's more than that?
Neha
I would say it's actually to help get to single source of truth by identifying and setting rules, for example, and being able to identify like, hey, where are some duplicates?. How do we eliminate those, removing edge copies, removing inconsistency. But that's not necessarily the storage layer either. You're just defining rules, right? The storage of this to integrate everything and have everything live in a single source of truth, that would actually be something like a lakehouse. So you want to have a lakehouse that either acts as a, again lakehouse is two components, right? Data lake as well as a data warehouse. You can have all of your data in a single storage layer. That's what you want to get to. And there's two ways to do it. We have a Databricks lakehouse, that's something that you can go with, or you want to have a lakehouse in Microsoft Fabric, right? So those are the two, I would say, industry-leading offerings for a lakehouse.
Samuel
And then the MDM will sit on top of those and you'll define the different rules to it, and that's where you'll plug the AI to get this data from, right?
Neha
Exactly. So you can bring it into the lakehouse, right? But then what you have is you have to build an architecture within the lakehouse that has your bronze data, your silver data, and your gold data. The bronze data is your absolute raw data. The silver data is after you've done a little bit of data engineering, you have removed, for example, all the duplication, some inconsistencies, things like that. And then your gold data is data that, all the table combinations you need to do, all of the different functions or measures that you need to calculate to actually show in a report the accurate information that you've now collected. And it's not looking at raw data. It's looking at the finest, most refined piece of data that's actually going to offer insights when you get to reporting. So that's sort of the stages that it goes into, and you want to govern and you want an MDM across all of this. So you want to have something that actually looks at that raw datand refines it. That's where the MDM comes into play.
Samuel
Okay. And Purview is connecting on top of MDM to give you the proper access, to make sure that the data governance is..
Neha
It's about maintaining a data catalog. So I would say there's so many components to the strategy, right? And part of this, let's say you're a technical user, right, and a business user will not necessarily know exactly what you have in this data file or that data format, this datasset that you've collected, right? So sometimes you want to make sure that across the organization, everyone can access a single data catalog, and they can just log in and figure out, hey, I want to look at sales data. Where would you go? Who has what? Where are we collecting information from? That's the kind of information that a data catalog helps to reveal. And then there's data classification. So again, to your point, sensitivity labels, what kind of data is it? Is there metadata that we can collect around this just to enforce again how and where that data lives? Where is that team? What's the team that, for example, has the ownership? So you're defining ownership, you have governance in place, and then you also have the visibility of the flow of the data. So data lineage is a big piece of this as well. That's where Purview really helps. So it's helping to solve this governance sort of mandate. And then anything to do with, again, arriving at single source of truth, the golden record, data quality, I would say is where master data management plays a role. You need both.
Samuel
Okay. And what's the challenge you're seeing the most from organizations trying to implement an MDM?
Neha
Yeah, I would say first thing I would say there, Sam, is probably that they don't have one, right? A lot of organizations will come to me and it's the same exact pattern of problems or challenges that they're facing: not having a single source of truth, a ton of integration work that they have had to put into place because of all the different sources they're trying to get data from, and then there's also, for example, not having a master data management, not having governance policies, not having a working group, not having that process internally to be able to get to that single source of truth essentially. So I think there's just not having those solutions, but also not having the processes in place as well, is probably the main challenge.
Samuel
Okay. You've discussed a lot about data governance, data quality, et cetera. So how critical is it to have good quality of data? I mean, when we're specifically talking about developing an AI model, what impact will it have to have bad quality or not have gone through all the different steps of going to the MDM, going to data governance, et cetera? So what will be the result of it?
Neha
Yeah. And I think, Sam, you absolutely covered it really well earlier too when you said garbage in, garbage out, right? So if we don't have a solution that ensures data quality, you don't have a reliable source of truth. You have no information there that you can actually trust. So even if you gain some metric, right, or you apply a function or you have some reporting that you're looking at, that's not necessarily the truth, right?. So I think the risk there is that you're not informed and you won't have the information that it takes to make informed decisions and to lead the way forward for your organization.
Samuel
Yes, makes total sense. Any best practice to share around data governance and data quality? I say that to people listening that don't know where to start. So what are the best practices? What should you look at first?
Neha
Yeah. I think it's getting together a working group or a set of stewards across the organization to essentially be like, hey, let's get together and have this as a mandate. We need to have a data governance strategy, right? We need to put together a bit of a working group. We need to have a meeting here where we can go through all of our datassets, we can specify individuals, Teams, where they sit, we can catalog some of this. We need to ensure we have access controls. We also want to get the training and awareness, right? So if you have a group of individuals that get together and define what that data governance strategy is for the organization, and they each have a role to play, that I think is absolutely critical to driving some of these practices forward. And of course the tools are there to help.
Samuel
What are the critical tools that you, I mean, we talk about MDM, we talk about Purview, but when you're building an AI solution, what are the tools other than the data? The data is the fuel of your AI, so I'll call it the first layer. But then when you're building the tool itself, there's so many tools out there. I mean, even from us, we're hearing about Copilot, we're hearing about AI Search, AI Studio. What should a company look at when building those solutions?
Neha
Yeah, I would say, so I'm going to repeat it again, data governance and MDM for sure. You want to take a look at Profisee, maybe CluedIn, Purview, right? So those are the three tools that come to mind from that perspective. And then when you're looking at just the data platform as a whole, you want something that is very much a lakehouse storage layer. So you want something that's inclusive of all of your structured data, unstructured data, semi-structured data. You don't want it to be sitting in siloed places or storage formats as well. So things like Microsoft Fabric, you want to look at Databricks as well as an option depending on what your needs are, and that's really going to help you remove those data silos and get all your data into one place so that all of those other solutions, like the MDM, the data governance, can actually go and help you enforce that single security model. And then from the architecture in terms of the actual AI use cases, the architecture very much depends, right? But the Azure AI services that I would actually urge customers to explore is Azure AI Studio. That's a really easy way to go and get your hands kind of dirty with some of the offerings and services that we have, and actually chat with your data. And the interface is really, really easy and intuitive. The other piece is AI Search. AI Search is absolutely critical for actually indexing the data, and that's going to be critical for the AI engine to use the data that you wanted to be trained on. The other piece is Cognitive Services, so that's anything like audio, vision, things like that. Then we also have Document Intelligence. Document Intelligence essentially is like PDF documents, like you were saying, Sam, or documents or faxes that you get that you want to make sense of and actually have tagged and qualified, right? Those are pieces that you also want to look at. And then of course, just even holistically when you're building that architecture, you're thinking about these technologies, you want to look at a RAG architecture, right? So that's something that I would encourage folks to look into.
Samuel
Maybe if you want to take a few seconds to explain RAG. A lot of people are asking me about what is RAG. But it's such an important concept in AI when you're using Copilot or even when you're using ChatGPT, which is basically using RAG on top of a Bing search to gather context. So can you explain RAG in simple terms?
Neha
Yeah. I would say, simple kind of sentence, you're retrieving data that sits in your traditional data sources or your external data sources, and you're essentially processing it through an LLM, right, that's been built, let's say a 4o, maybe you're using o1 now. But essentially you're using the context from that data source, you're using the LLM processes, while you're generating an answer based on the blended sources. So it has been trained already on all of the information that's available on the internet, but you're also giving it context about your source. So the three main stages of the RAG architecture are data prep, its retrieval, and then its generation. So that's how it works. And the key component of all of this really being able to index your data correctly. So you actually want to be able to use something like AI Search, for example, to index your datand actually get it ready for RAG. So it's known for retrieval-augmented generation, right? So you're using the relevant data there that's important, and you use it to augment context for the LLM to actually process your request.
Samuel
Super well explained. Thank you for that. You mentioned a lot of Microsoft products that are on Azure, right? So that's our cloud offering. So how important is cloud for an AI strategy? I'm hearing a lot of customers asking, can I own this on-premise? So what's the pros and cons, and why would you suggest to go on a cloud-first strategy?
Neha
Yeah. For, I would say, one-off use cases, it's absolutely possible to do something on-premise and build something. I think the challenge there is going to be scalability. So what are you going to do as data grows over time, because it will? And so you're going to need additional storage capacity, you're going to have to scale that entire solution. But I think going to the cloud there and thinking about that at any given stage is critical because, again, if it's not feasible from the broader strategy, one-off use case is totally possible. From a broader data strategy, it's actually not feasible to remove those data silos and to decrease the integration challenges that you might face without migrating to a cloud platform, right? So that's the reality. You need to have that scalability, you need flexibility, and you need to have access to those advanced solutions that you're trying to get to so that you're prepared for when that data continues to grow over time.
Samuel
You just mentioned scalability and flexibility, right? So how can entERPrises ensure that it's scalable and flexible? I will assume that if you're building a model or if you're building a solution around AI, you will get more and more adoption, right? So you'll have to scale at some point.
Neha
Yeah. I would say exactly like you said, Sam. So you want to have the right technologies that can scale as your data grows, right? And then identify some use cases not only for today, but for the future, right? So making sure you have flexibility now and from the data platform or the solutions that you're investing in, you want to identify use cases for now, but you also want to identify use cases for, let's say, three years out. A lot can change in three years, so by no means is this writing it down and going, yes, we need to check this off, but it gives you an indication for what this data platform needs to allow you to do to be able to scale with you as your needs change. And then the other piece there is investing in open standards, right? So traditional technologies actually have essentially the data that you have or the data platform you're using, let's even take SQL as an example or Oracle, right? There's vendor lock-in for these platforms because you have to know Transact-SQL, you have to know specific languages for traditional technologies, right? In open standards, it's a little bit different. So you want to have a lakehouse like Microsoft Fabric, like Databricks, that supports interoperability, right? And it's going to help you prevent vendor lock-in. And the data format specifically here that Microsoft Fabric and even Databricks as well supports is Parquet and Delta Lake, right? That means your data when it lands in Fabric is automatically converted into a Delta Parquet format, which is open source, right? And it's compatible with a wide range of tools, compatible with a wide range of systems, and it supports things like ACID transactions. So that's super, super key. And what this means is if you ever want to pick up, take your datand change platforms or vendors, you can, right? That's what we're enabling.
Samuel
We've discussed a lot of the challenges, right? Implementing an MDM, implementing the data strategy, defining the maturity. Now globally when implementing an AI solution, what are the common challenges you're seeing organizations facing?
Neha
I think I'm going to tackle this a little bit differently, maybe as advice that I can shape up from the experience I've had. But essentially, it's to assess where you are. I think that's a really key starting point because the journey that you're on might look very different from your peers and your competitors. And you might be at a stage where you can actually start implementing things like Copilot, sure, because it's a solution that security is taken care of. Copilot is something that is fully managed by Microsoft, right? But it doesn't necessarily mean that you have the in-house expertise to start building custom solutions. That's a different kind of AI that requires a different kind of AI maturity. And then go back to, I would say, even like Dr. Lippitt's model, right? So identify where you are, where you want to go, what's the gap, and how can you have SMEs or experts across your organization that can actually start to get there and take a little bit of ownership on themselves to drive the organization forward. And then invest in skilling and training, right, across each of those. So the more you know, the better decisions you will inevitably make. And a lot of times I'm sitting down with organizations and I'm realizing we haven't covered the basics. We're talking about the technology here, but we've never actually talked about the main concepts. We need to review them. We need to go back to the foundation to be able to understand how the technology helps and where perhaps it's lacking, right? Where we need to have certain other considerations because it's not just the technology that's going to help address everything that you're looking at when it comes to getting to an AI mature organization. And then the last piece there is, is there something that you have to automate? Is there something you're trying to achieve?. What is that pain point for your organization that you want to address? Don't start with the technology in mind. Start with the problem in mind, right? And these are things, again, everyone has thought of for almost every other time in history that we've had a change, but I think it's absolutely even more critical now than ever before.
Samuel
Thank you. Thank you so much. So there was a lot of information. That's really enlightening. I mean, I learned a lot today. I haven't heard about the MDM concept before today, so that's really interesting. And I think people listening will have a lot to consider. And I hope we'll be able to do another episode and dig deeper inside those concepts. So thank you so much for coming in today. It was really interesting.
Neha
Thanks for having me, Samuel. I appreciate it. It was a pleasure speaking with you.
Samuel
Awesome. Thank you and take care.
Neha
Thank you.
Samuel
Thank you for tuning intoday's episode. We've learned a lot about the journey toward AI maturity, from assessing data readiness to implementing strong governance strategies. It's clear that organizations need a solid foundation to truly harness the power of AI. A big thanks to Neha Sharma for sharing her expertise and valuable insights. If you enjoyed this conversation, be sure to check out our previous episodes, Unlocking the Future of Customer Service with AI with Marcus Schmidtt, principal program manager at Microsoft, and How to Leverage Copilot for Sales with Mathieu May, global black belt at Microsoft. We'll be back soon with more AI discussions, so don't forget to subscribe. See you.