Balancing Innovation and Regulation in MedTech Development with Karandeep Singh Badwal | Ep. 35

Hi, welcome back to the Med Device Cyber Podcast. Today we have a guest Karandeep. We're going to be talking about regulatory affairs, quality management systems, and some of the differences that have come out recently across the global regulatory affairs perspective. Karandeep comes to us from the UK. I'm your host Christian Espinosa, and I've got the co-host here today, Trevor Slatterie. I think one of the podcasts we did before, I wasn't here. Trevor handled it by himself, but he did a good job. So, how's it going, Karandeep, today? I think you're coming from the UK today, right? That's correct. Thank you for having me. Awesome. Do you want to give a little bit of background on what you do and your organization does? I know you're also a podcast host. Maybe a little bit about your podcast and how many years you've been working on that, and just a little background on you. Sure, so I'm Karandeep, and I'm founder of QR Medical, a quality and regulatory consultancy working within the medical devices space. In terms of my background, I actually still come from a pharmaceutical background, which was what my education was based in. So I did a bachelor's in pharmaceutical science and a master's in pharmaceutical quality by design. But it just so happened that when I left university, the first job that I had was actually working within medical devices, and I just kind of kept it going from there. I started working in traditional devices with things like pacemakers, defibrillators at St. Jude Medical, which eventually became Abbott. Over the past four to five years, I've been doing a lot more in the software, AI, and machine learning space. That's where it seems that the medical device industry is moving forward, not just in other industries as well. Also, outside of that, as you mentioned, Christian, I'm host of the MedTech podcast, which I started back in 2021, and I interview different MedTech leaders, founders, CEOs, experts, etcetera, in the field, and give them a platform to share their story. I'm also a content creator on LinkedIn as well. For anybody who does follow me on LinkedIn, a lot of people find quality and regulatory often very confusing, and there's a lot of guidance out there. If you were to go out and read the regulations, you probably woke up more confused than when you started. So my point of the content that I make is just trying to make things a little bit more easier to understand, effectively. Awesome. Well, I find quality and regulatory a little bit confusing as well. We had someone on the podcast before that described regulatory as offense and quality as defense. Would you agree with that statement? Yeah, there is some element of truth to that because with quality, it's often about taking a proactive approach. For example, the biggest tool that we use often in quality is often the CAPA, corrective action, preventive action. A lot of companies out there take corrective action; they wait for something to go wrong and then try to fix it. What quality often tries to encourage is taking a preventive approach. Look for something that may potentially go wrong and put something in place to make sure that thing doesn't go wrong in the future. So, yeah, I would somewhat agree with that. Whereas regulatory is more about getting that market approval, getting the product out there, and filling that, whereas quality is more like a process on the whole thing. So regulatory is for your product, where quality itself is on the company as a whole, and quality really is not a department; it's more of a culture. What, I know you mentioned you're doing a lot of work with AI and software as a medical device, which we see as well, the industry shifting that way. What are some of the biggest challenges you've noticed working with your clients from a consulting perspective when they have software and some sort of AI model? The issue that I've often found with companies, so let's, for example, I bring them on as a client, and they're at version 15. When I try to go back in time, it's, okay, what did version one look like? There were never any proper design controls in place. So, realistically, they don't really know the changes that got them from V1 to V15, or in some cases, they were not the high level of it. But how did that affect the code? Can it give false positives? Can it give false negatives? What are the risks that come with it? So, often, just trying to do things like that retrospectively can be difficult. And the second one, Christian, again, timely to the podcast, is of course, called cybersecurity. You know, they all realize that there needs to be some cybersecurity to it. But then, how far do you go? To what extent do you take it? How much cybersecurity is needed? Or in some cases, where companies have done cybersecurity things like penetration testing, but it's probably done like a version that's two to three years old, and now the software has changed so much where they now need to do that level of testing again. And then the company's thinking, okay, great. Does this require a full test? Does it require a partial test? So companies often out there don't really know because even if you were to go out and read the guidance documents from the FDA, etcetera, it would tell you that you need cybersecurity, but to what extent, to what sort of testing, and how often does this testing need to happen? These are the questions I often get from clients. Yeah, that's an area that we navigate quite often, and a lot of questions come up because the requirement is like typically people say once a year, we need a penetration test. But as you said, if you've gone from version one to 15, and there's significant changes, you probably needed 15 tests, and if those changes went in in one year. What is your perspective, Trevor, with what Karandeep is saying and our clients coming to us about cybersecurity, like in the versions, and do they actually have evidence of the changes between the versions from your experience? Because you're more on the delivery side than I am, obviously. I think it's perfect timing for this question to come up. Right before we started recording this, I was actually helping a client with this exact question, and that's, where do we draw the line between what is a cybersecurity-related change and what is not? And it, there's a little bit of ambiguity. I know the FDA actually just about a week ago published some new updated guidance that provides some clarity on what they're looking at as far as changes that may impact cybersecurity. But it's not an exhaustive list. The examples that they provide are things that will affect the authentication mechanisms, things that will affect encryption or how data is transferred or processed. In general, those are really good examples to go by, or any significant alteration of the device functionality that's likely going to need to have some more penetration testing around it. I think where we can see companies getting away without doing additional penetration testing is minor changes, things like cosmetic tweaks or small changes to an algorithm if they have this, you know, AI-enabled cloud platform. If it's not affecting the way that a user can interact with the device, which would be how an attacker interacts with the device, the cybersecurity risk and the cybersecurity modifications are likely going to be a little reduced. Agree. And one of the areas that I think is challenging is with AI, it technically is not cybersecurity, but I feel like we end up talking to a lot of clients and prospects about this: how they train the model with AI. From your experience, Karadeep, is this something that you feel like the industry has a good handle on, like the data they get to train the model, or is this still in its infancy? In my experience, I very much say it's in its infancy because how do you validate whether that data is good or not? That's the issue with AI. You can have two of exactly the same products; one has a good data source, one has a bad data source, but on paper, they will look exactly the same. The difference is the one with a good data source will have a lot less bias, will give a lot less false positives, false negatives, will probably give better results in comparison to the one with the bad data. But again, when it comes to training, that's one thing that I find, especially from a regulatory perspective, is not really looked into. What is the validity of the data that they're using to train that particular model? Do you feel like the regulatory authorities have a decent handle on making sure the model has the proper training, or are they really, I guess, putting that responsibility on the manufacturer? I think it's very much on the manufacturer. I mean, if you were to look at regulatory bodies, you know, FDA, etcetera, how many people out there actually understand AI, machine learning software? The chances are going to be very, very few. So what they often do when it comes to regulatory is often a documentation check. They're not really playing around with the device or testing the data, seeing really how it works. They're just relying on the documents that have been given to them and using that for a regulatory approval. So again, if it looks great on paper, then that may well be the case, but there may not be that it works well in reality. And from a cybersecurity perspective, Trevor, what are some of the things that have changed that we have to look at when there's software as a medical device with AI, for instance? AI is still such a new technology, and some of the cybersecurity threats that are being discovered in it are coming out at a pretty astonishing rate, in the same way that any new technology, a lot of risk is going to be discovered. So trying to stay on top of a rapidly evolving technology where the attackers are trying to understand it just as quickly as the defenders, and each one of them sort of becomes a cat-and-mouse game to try to figure out who's able to push the security research forward the fastest. So I think that's a little bit of a unique challenge that comes up every once in a while. I know a recent example of that was around 2008, 2009, when smartphones were becoming far more prevalent. Every medical device had a companion app attached to it, largely for marketing purposes, to say we have a medical app. Then the FDA started to clamp down, started to regulate these as medical applications. And so now all the regulations were enforced as far as how you're handling security in this application. But the threat landscape was running wild at that point since it was just such a new concept, and people weren't really sure what could go wrong there. So I think we're seeing a similar case with AI. And with quality, I know Karandeep, you do regulatory and quality, and QM is part of what you do, and ISO 13485. Where do you see, like kind of what you mentioned earlier, like version 1 to 15, that there should be the design history file in the QM. Why is there such a gap? Is it an awareness problem? Is it a funding problem? Like, and we have the same thing with cybersecurity. There's this big gap; people don't really think about it until later. From your lens, what do you think the reason is for that gap? So my view is when they start the software development phase, the last thing they think about is quality or regulatory, and they start thinking about the quality management system when it comes to a time. You think, okay, great, we're a medical device, and we want to get regulatory approval in a year or two from now. But by then, they're already on version 7, 8, or 9, etcetera, and they've built that without a quality management system. So there never was really a design control process in place at the time in terms of how to do it, or if there was a design control process in place, it wasn't really to the standard of which is required under 13485. So I think that's really where the gap comes into it. But then my view here is what I always say to companies is, in your early stages, just have a partial quality management system. You're going to want document controls. You're going to want record controls. You're going to want design change history files, etcetera. But these things aren't just limited to medical devices. You want document controls. You want to have records. You want to record changes. You want to see how changes are made. So my recommendation for the listeners out there, if you're in your early stages, just adopt a partial QMS. It doesn't have to be a big fancy system; it could be paper-based, but just have some sort of change, and then ideally try to follow 13485 principles. So when you are ready to have a full quality management system, it's more of a seamless transition into it. When do people typically engage with you? Probably too late to hear your advice you just gave, right? I mean, ideally, I should be there at the start. But then what do companies do is they think, hey, great, this is it. We've got this product. It works. It's fantastic. We've been to conferences. Everybody likes it. Let's get regulated. And it's just not as simple as that, right? And they're like, and particularly with companies, what do they try to do? And you guys have probably seen it as well. Our product has 50 different claims. It can detect 50 different types of cancer. And I'm like, guys, yeah, it doesn't really work like that. You can maybe have two or three different indications, and it's also dependent on your clinical data as well. Though often they come to me at the very late stages, but ideally, they should be coming right at the start. And then what I often say to companies, if you build quality and regulatory at the beginning, it's actually cheaper in the long run. Trying to fix something and do things in retrospective, like the example I was mentioning, like design history files, etcetera, it's somewhat impossible or very difficult to do, and there's always going to be gaps. And then sometimes trying to patch something is a lot more time costly than it is to just do it properly from the start. So the answer to the question is ideally yes, they should be there at the start, but they often come at a very late stages and expect me to work some sort of magic and somehow get it through. What do you think, Trevor? That similar situation for us. Yeah, I was going to say that sounds about exactly like the way we answer that same question. So, well, I think from an innovator perspective, I try to look at the world through other people's lens. If I'm trying to come up with a product that I don't know anyone wants it, you know, it's an MVP, the minimum viable product, why would I care about regulatory or cybersecurity when I don't even know if this thing is a good fit for the market, right? I mean, that's probably the dilemma they're facing. So, if you're talking to an innovator, how would you address that perspective? I'll start with you, Trevor, then I'll go over to you, Karandeep. I think it comes down to having your technology, having your product. If it's something that you're creating because you see a market fit, you want to prove that market fit. And if you're able to do that, if you're able to prove you have a market fit, if you've covered all of your bases, and you do actually have a good product, then you want to ensure you have your quality all the way going back. Similar to quality from the actual quality perspective, I think cybersecurity should typically be thought of as quality in software or quality in a product. Safe products are good products. They're high-quality products. So assuming that your product does have this market fit that you see and you will be able to convey that to potential investors, to potential customers, then you want to make sure that you have the best version of that product that you can put forward, and the best version of that is going to be a secure, high-quality, safe product. What do you think about that, Karandeep, that cybersecurity is part of the quality? It should be, really. It should be part of your quality management system. It should be part of your procedures. Every time there's a design change, for example, under 13485, there has to be some sort of impact on the quality management system. There should also be assessment on risk as well. Cybersecurity is also part of your risk management. So yes, cybersecurity really needs to be involved within your procedures, and also people need to realize that, particularly software developers, have you guys ever tried to tell a software developer what to do? They probably don't like that in post-editers, right? So, yeah, you've definitely experienced that. They're fantastic at what they do. Scientifically, they're fantastic at what they do. They're very great at their job. But you know, trying to tell them to work in an agile fashion or a design change, they're used to, I want to add this feature, change a couple of buttons in the code, and press enter. In the medical device world, we can't do that. We need design inputs, we need design outputs, we need design reviews, we have to record everything. And sometimes software developers think, hey, you know, this is just a small change. I just put it through. And then somebody like me in regulatory comes and goes,