How to Achieve Digital Healthcare Transformation

Sarah: All right, good morning and welcome to our webinar, How to Achieve Digital Healthcare Transformation my name is Sarah-Lynn Brunner and I will be your host today. As you we can see I have a couple of guest speakers. I like to introduce Ed Hine whose manager of digital verification and validation at LifeScan, a Johnson & Johnson Company. We also have Eli Snell, Senior Director of Firmware Software Applications at Proteus Digital Health and our moderator today is Avery Lyford, Chief Customer Officer at Apexon.

Before we begin let me remind — review some housekeeping items. First, this webcast is being recorded and will be distributed via email to you allowing you to share it with your internal teams or watch it again for later. Second, your line is currently muted. Three, please feel free to submit any questions during the call by accessing the Q&A button, the navigation, the top center of your screen. We will answer all questions towards the end of the presentation.

We will also do our best to keep this webinar to the 45-minute time allotment. At this time, I’m going to turn this presentation over to our moderator Avery Lyford.

Avery: All right, thank you. Let me first, a little background on some of the presenters here is really interesting, interesting backgrounds. Ed today is working at LifeScan, but I think his background is very interesting. He’s worked with some very complex, complex environments such as the managing the quality of places like Honeywell and dealing with issues of continuous quality and very complex, complex processes.

Currently, LifeScan is responsible for doing the blood glucose measurements. It’s really leading the way in terms of how you can start to use technology to improve the experience for people with diabetes conditions. Eli at Proteus, Proteus is pioneering a whole new category where you’ve got a pill that can communicate back to your — communicate back to your doctor and creating that whole new category. He worked previously in some very complex areas such as St. Jude Medical doing firmware for things like implantable pacemakers as well as cord stimulators, spinal cord stimulator.

With that let, me kick off and do a couple of opening questions. One of the things people talk a lot about digital transformation but it means a lot of different things to different people, Eli and Ed. First Eli, what’s your definition of digital transformation? How do you think about it?

Eli: Hey, thanks Avery, at Proteus we think about digital transformation as transforming the healthcare industry. We’re carving out a new category called Digital Medicines. We’re working closely with healthcare providers and pharmaceutical companies to transform what has been fundamentally an analog space for decades and move the practice of medicine including the pharmaceutical prescription process as well as the ingestion of tracking and monitoring into the digital world. That’s a tall order. We’re making great progress, but it’s a good journey down that digital path.

Avery: Then, Ed, how do you think about it because at J&J really different situation. You’ve got a huge set of incumbent companies and partners that you’re working with and you got to bring them all along as you make this make this move. How do you think about digital transformation?

Eli: Can you hear me, Avery?

Avery: I can now.

Ed: Okay, yes, sorry mute button on the computer. As part of our diabetes care franchise company what we’re doing is enabling patients to track their blood glucose readings on their mobile devices and online and their health care providers and health management companies can access their data via API interface off of our website. It’s faster to access the data, more accurate tracking and trending and being able to present that data to the patients, their providers and loved ones more accurately so that they can live a normal life. As part of — as you mentioned we’re part of Johnson & Johnson, a larger company, we leverage some of their enterprise software in the things that we do.

But, are somewhat independent and autonomous in terms of moving quickly, releasing new releases quarterly with new features and the feature-rich applications to help our patients and their loved ones and health care providers manage their condition more effectively.

Avery: I’m curious, Eli, when you guys — what’s been your biggest surprise as you’ve gone down this path of trying to create this category and working with the healthcare industry?

Eli: Avery, we found out that when patients are using our platform, when they’re using our product it works. We have great data showing the outcomes are improving. However, I think our biggest surprise and it probably shouldn’t be a big surprise but the idea of trying to change a healthcare industry is certainly one of our biggest challenges. Thinking we can go into a health system and expect doctors and nurses and other practitioners to change their day-to-day workflow, to use new tools that they may not be familiar with that’s been our biggest challenge.

We’re working hard to address those in terms of the service model and we’re trying to fit in to existing processes before we can then hope to change them. We had some early product experiences where we put our product out there expecting to do point of sale distribution directly to patients. Sounded like a great idea, but it turned out that that was too different, too new and we needed to start with meeting patients, meeting doctors, meeting healthcare practitioners where they are.

Then working with them to change rather than going in with a fully digital solution from the beginning.

Avery: Yes, they’re the same with revolutions first they laugh at you, then they fight you, then you win, right?

Eli: Exactly, yes.

Avery: Eddie, you’ve got a little bit of a different challenge because you’ve got a whole set of existing customers and relationships that you’re looking to move digitally. Have you had a similar — what have been your surprises as you’ve made that journey?

Ed: Yes, I think part of it is educating the consumer. The one thing I didn’t mention in terms of digital transformation is taking your mobile device from just a monitoring tool and tracking and showing logbook entries and trends and patterns to an actual Class III medical device that can start making decisions. One of the goals, the change they had was developing this artificial pancreas.

I know there’s clinical trials going on right now with one of our competitors where they’re actually using cloud-based algorithms and technology to do calculations and manage people’s diabetes automatically so to speak. That’s just getting people A, to rely on the app more for the information. Then even taking that next leap of faith where that you’re putting that logic in your hand. That’s been a challenge for us just from regulatory perspectives and also development obviously.

Avery: Reminds me when you’re learning to fly a plane one of the big shifts is going from — going to where you just go to instrument flying where you completely depend on your instruments. You can — you no longer– you have to trust that what the instrument tells you is the truth in terms of where you’re located. It’s I think the similar thing started trust that the technology will actually give you accurate readings even if they’re not what you expected.

It strikes me with the same shift mentally where you have to be able to have just complete confidence in your instruments and your data. Eli, what have you seen with the doctors have they started to try to adopt this technology and the consumers and what were big positive surprises where people said this was a lot better than what they expected?

Eli: Some of the rewarding parts about working in the medical device space is those stories about the patients that have been helped by the products. I saw that in my past life at St. Jude Medical and I was getting a chance to interact with pacemaker patients who are now able to carry on with their normal lives.

We get to see that at Proteus too. We get to hear from patients who have either been directly affected by moving towards whatever goal they might have, whether its improved blood pressure or improved heart failure or improved treatment of hepatitis C. Really importantly, we also hear from family members who no longer have to worry as much about their loved ones. Similar to what Ed said we have the ability for family members to participate in the care.

Then we hear from doctors as well who are spending less time, who are focusing on the patient’s that need the help focusing on patients that need the help more than spreading their attention across all their patients. They now have a tool where they can more quickly triage their patients and have a view through our web portal to see which patients need help that day. It gives them an ability to prioritize.

Avery: Ed, one of the things that people sometimes say is that working in regulated medical space. It’s really difficult to use technology because you’ve got all this regulatory environment around you. What’s been your experience and how is automation and some of the test automation made a difference in dealing with the regulatory agencies?

Ed: Well, I think from my perspective, you mentioned some of my prior experience working in bio pharmaceuticals and distributed control systems with analytical instruments. The mobile device space is completely new to me for the last three years or so. I was intrigued and inspired by some of the automation that’s available more so than what was done on distributed control systems what I’ve experience there.

It seems like the FDA, really can’t keep up and is unsure on how to regulate things that go into a mobile device. There’s been a panel recently that J&J is a member of. I think they selected 12 companies to actually prove out their quality methodologies. I forget what the name of the program. I should know this but it was the fast track or pre-approved mobile apps that are being developed. Unfortunately, we’re not part of that but I think that the systems that was implemented with Apexon is help using Perfecto by Perforce and even the UFT part of HP Quality Center.

Testing our website and those things just lend itself towards building quality into a system and quickly assessing the quality of changes and performing regression testing which just builds a case to an agency that we’ve done on a thorough due diligence and testing our product.

Avery: What’s been your experience because one of the things that is always a challenge as regulators one other stuff is repeatable. How is the testing helped in that process to help move things towards approval and inspiring everyone’s confidence?

Eli: One of the things we began about three years ago was a shift from our more traditional waterfall approach to development. I think there were people at that time that would have said we are operating in an agile environment but when we look back, there were a lot of elements that were still in a more traditional space. Long tales of testing at the end of a development, we’d see upwards of six weeks of software verification testing after code freeze. We’d see manually created documents like requirement specs, test reports.

We’ve invested quite a bit, really in the last two years on continuous integration to ensure that we have a much smoother built process. Then on moving our testing into an automated system, some of that is required build-up of infrastructure. Some of that is required using off-the-shelf tools and building some software glue to ensure that we have a fully integrated test environment. What we’ve seen now is we test as we go. Huge component of Agile is bringing work to done including testing in every two weeks spread. Then when we get to the tail end of the program, we’re able to really shrink that time from code freeze to release. We had a release that went out just yesterday and the time between the last bug fix and release was about two days compared to six weeks or longer from a while back. When we then had subsequent regulatory reviews, whether it’s FDA or whatever notified bodies in Europe, we can show them that we’ve performed testing as we go.

We can show them that we have robust test protocols. We can invest more time now in building up the automated test suites because we’re saving the time from the end of the program and that also helps us in the subsequent iteration. It’s been a big win. We talked a lot at the beginning of this webinar here about the shift to digital for our customers, for our products. A big part of that story is our internal shift as well, moving from word, moving from paper forms into digital built systems, digital test systems.

Avery: What will be your biggest advise for people if they’re trying to do that? What are the pitfalls and roadblocks because the challenge always is how to conceive of a different way of doing business because you said you thought you were good and now looking back and you’re like, “God, there’s so much progress that we’ve made.” What’s a reasonable expectation? How do you set targets to get people move?

Eli: I think there’s two ingredients. One is you have to have good people. You have to engage with either a good partner or you have to hire the right people that are domain experts in areas like continuous integration, continuous deployment. It can’t be a part time job of some engineer or some engineering manager to build up that level of automation that really has to be a focus for one person or a small team of people.

It’s the right people across those areas. The tooling, also the automation test whether it’s the frameworks or the test protocols themselves. People and the focus is its ingredient number one. Management support is ingredient number two. For those familiar with FDA regulations on quality systems, one of the first things they say for design controls is you have to have management support. That’s really key for the shift to automation. The shift to continuous integration is you have to have management support, whether it’s just the budget support to hire folks.

Probably, more importantly it’s their support for changing the way that we’re working. Changing the way that we’re thinking about prioritizing work. Changing the way that we’re thinking about packaging our results up into documents for various FDA artifacts. If that management support is not there, it’s going to be a struggle for the teams.

Avery: What was it that gave the management support? You had to show them it was purely mandate or you were able to show, “Hey, if you give me the support, I can deliver you these kinds of results for you?”

Eli: It was a combination of. We brought in some new leader in the organization that had more familiarity with working in Agile teams and working with fully automated consumer products. For them, they brought that expectation in. Expectation of, “Of course, we should have this automated.” “Of course we should be working in this way.” They were able at that more executive level to align across their peers.

It’s also from the grassroots. I remember we had some project managers and scrum masters that just push hard. They were really passionate about moving towards Agile, moving towards automation. They just kept pushing and it took some time to gain traction. I think a combination of that grassroots push and the right new management folks that came in naturally. We were able to be much farther along in our journey. I won’t say we’re there yet, but it’s a journey. We’re working through it.

Avery: Ed what was yours? You came from a really different environment in terms of the industrial systems and some of the control systems there. I think in the past we’ve talked about the shift to being that — it’s mobile app to a very different world because the consumer expectations are different and the FDA expectations are a little bit different. What were the big challenges you’ve seen? What would you go back and tell your younger self if you could about what to think about and what to do as you’re implementing all this?

Ed: Yes, I would repeat a lot of things that Eli said. Obviously, you need management support for any amount of change. Maintaining enough in-house expertise to be able to make intelligent informed decisions and then identifying the right tools and vendors to execute it properly is one thing. What I would tell my younger self is expect change and change before you have to, know that things are rapidly changing particularly in the mobile device area where technology is advancing quickly.

It was easy for me coming into this role because it’s a new space so I was learning, seeing everything’s so new and fortunate also that I had good, talented, knowledgeable people here on staff as direct employees and being able to find or identify good qualified vendors and tools as I said are important. Again, to reiterate some of the things that Eli said having management support is really key. Having good passionate people or scrum master is key as we are in this agile space as well.

It’s funny because when I talk to some of my colleagues in the distributor control system and GMP biopharmaceutical R&D environment and talk about agile. It frightens them because that fast pace in development and running quickly like that in scrums is probably not something that would work in that environment. It obviously works well here in the lower regulatory environment that we’re in a particular as the class two device.

Avery: Ed, it’s interesting you talked about the different groups J&J has got a bunch of different cultures in terms of expectations and also different regulatory environments. How do you start to shift that and how people understand that what you’re expecting there in terms of super detailed specs and what are full process and that level of detail isn’t appropriate for the agile mobile world.

Where you’ve got — as someone was telling me yesterday you’ve got Apple coming out with innovations but they don’t really care about whether or not you want to adopt them. iOS it’s coming along and you will be getting facial recognition whether you want it or not instead of finger prints. Sometimes you’re forced to march along whether you want to or not innovations, how do you deal with all that?

Ed: You’re right the fingerprint ID for Apple iOS is something that we just implemented on our app in our one of our latest releases. Now, that’s going to be going away and moving into facial recognition. Even changes on the automation front with iOS 11 that we’re having to adapt with. It’s never ending, but then to your point talking to different colleagues because one of our consumer groups has developed this seven minute workout app with development group that we’ve actually used in the past. Obviously, there’s limited regulatory intervention there. Fortunately, it’s incumbent upon our quality assurance group and regulatory groups and me as a test leader to make sure that we’ve got the proper test artifacts and things in place and procedures in place.

Wrapping up automation is just a key part of that and trying to get that automation up as high as it can go safely. There’s always some test you want to run manually. We wouldn’t want to be much higher than 80% fully automated. Again, the speed in return are much better for that. You can automate a lot more. Some of the automation it’s all in the distributed control system environment was very manual, where we’d actually use the control systems to power down modules and test redundancy and that sort of thing.

It was set up more like a telephone switch gear. Heavily programmed automation whereas the — there’s a lot more soft touch automation with Perfecto and things that could be done here.

Avery: Proteus has been digital from its birth and certainly everybody there understands the nature of transformation but you’re dealing with a lot of incumbent and as you’re building a new category, how do you get all the different parties lined up? You’ve got everyone from Bluetooth to smartphones to cloud on the technology partner side, on the customer side. You’ve got a set of different doctors, nurses, healthcare professionals getting engaged. You’ve got to manage across that entire chain and get everyone — your rate limited by the slowest persons. How do you move that all forward?

Eli: As I mentioned before, when we do get the product on the patient, the wearable on the patient, when we get the digital drugs to the patient, when we get them using our app it works and it’s great. One of our biggest challenges is all the friction points between let’s just say now when the next patient coming on board. That friction points is anywhere from introducing the product to them in a clinical setting.

Maybe it’s a doctor who recognizes a patient might need or might benefit from a few months of use of the produce product or distribution. We’re not available in your corner CVFs, we need specialty pharmacy or looking into in clinic specialty pharmacy. Some of our health system partners have their own pharmacies. That’s different for the patient, that’s different than their corner drugstore. To answer your question Avery, it’s about finding a few partners that want to work with us.

Our chief commercial officer says we’re going to go deep and not broad. She’s picked a few partners, a few health system partners that have strong executive support that are willing to meet with us regularly and share feedback. Then back to the Agile, we need to be able to adapt quickly. If we learn that there’s a way to save time or get this information to the right person more quickly, we need to adapt to that and do it. It can’t be on a one or two-year roadmap.

It has to be done now in this release or maybe the next quarter. It can’t be two years from now because the partners aren’t going to wait that long.

Avery: It’s probably key being able to show — it sounds like — and having seen the product myself. We actually see it in operation, it can be hugely inspirational. Your mind can start to run wild in terms of the really useful applications. Like your parents don’t have to remember — you don’t have to worry about whether they took their medicine or not. There are some really interesting applications.

It’s that ability to show that end state at least a couple cases tend to get line everybody up because they all line up behind. Yes, that is the outcome we all want. Is that the key?

Eli: It is and we call that our magic moment. That moment when a patient first swallows one of their medicines with the adjustable sensor and then sees the icon on their app. With a check mark or a filled in pill to show that, that particular sensor was detected. Having been a guinea pig on that in one of our own clinical trials, it was sanctioned. I wasn’t on my own but it is a really magical feeling to — after working on this product and then they actually use it, and see that when you swallow something and shows up on the app, it’s pretty amazing.

Getting that story out to patients and family members has been important for us. We’ve been fortunate to have a fair amount of publicity through some of our engagements with health systems. We still recognize that there is that friction and one of our biggest focus areas is on that service model, working through anything from contracting, through distribution, supply chain, education. These are all things leading up to patient use that I don’t think we expected to be quite as challenging as they are.

They’re really important and we’re learning that now we’re making great strengths. Some of the recent product improvements are getting things like patient education, portals available so we can save a half hour of a doctor’s time. We know they have about five minutes, so saving a half hour is not trivial.

Avery: Yes, I can think of the test for a moment where you hear about electric cars and the first time, you get a new one. I think the same challenge that they had where you also need to build out this whole network of electric chargers. All of a sudden everyone’s going electric. It’s interesting to watch these shifts occur. Ed, one of the interesting things in the whole diabetes care is this shift toward the app has become not just a tick box item but become actually a driver for a lot of usage and adoption. It seems like a pretty big shift in that whole industry, is that true?

Ed: Yes, absolutely. I think our management is realizing that as well. We’re actually shifting our business model to more marketing patient data through our back interface, and seeing if there’s a potential revenue stream there as well. It used to be nice to have freebie that you threw in. Now, it’s becoming an essential that most competitors — most of our competitors and most people in this space offer it as an add-on to the meters or pumps or any other items that they’re marketing.

It’s essential needs to be clean, usable interface and we’ve made great strides this year. In revamping ours and producing new more WYSIWYG versions of our app. Through running that testing using the more automated means has helped us to advance it much quicker.

Avery: Is it the same kind of a magic moment when people all of a sudden realize that they can get much more continuous feedback and stop worrying and much about glucose levels and be able to get more real-time feedback?

Ed: We do tend to run sessions with consumers when beta releases, we get their feedback. Many of the bugs and fixes that we implement are coming from our tier three customer support database. It is great when we do point releases on a quarterly basis. When they come out and you actually get comments we have a knowledgebase that supports it but we’ve seen comments and people love the app and say, “Thank you for fixing this problem.”

I put in this trouble call or something a year ago, so we’ve had good interactions from that standpoint and we know that our customers are using it. In fact, we just won an award in Spain for digital solutions that was posted.

Avery: It’s interesting to me, especially in a medical space you have the challenge of the regulation but also you have the inspiration, you see the impact on people’s lives of the technologies you’re developing. Does that, as you’re dealing with this evolution, how do you take advantage of you getting the feedback, you mentioned to some of the customer feedback with the call center? How else do you get feedback to make sure that you’re on track from a quality standpoint? What other channels do you use? How do you know that you’re doing the right things?

Ed: That’s primarily through our customer service and marketing groups. The marketing groups they’re mostly concerned with expanding to new countries in different markets but also offering new features, bolus calculators, food databases, things that are key. Again, some of our competitors are already offering that we’re integrating into our app and have tried to integrate in the past those are important things that they get through marketing surveys and that sort of thing.

Avery: Eli, what has been the biggest — the thing had asked Ed before is if you go back and give some advice to your younger self, what would the advice be about how you would do things? How will you get things to move faster and what would you do differently sooner.

Eli: It’s a pretty similar answer to what Ed gave which is it’s never too early to start some of these initiatives whether it’s that transition agile or whether it’s hiring the right expert in the right area. We thought we could do some of these things ourselves for a while. I remember we shifted into the Atlassian Suite using JIRA and Confluence probably two years ago. I remember a very short fantasy amount of time maybe a day.

I thought I could do that or someone in my team could do that and that’s a crazy idea, you need experts, you need people that can focus on their domain whether it be tools or building up our automation or being an agile coach or being a dedicated scrum Master. If I could rewind a bit I would have hoped to hire those people on day one not two years into my time here at Proteus.

Avery: Maybe comments about the things where Apexon has been helpful to you in that journey.

Eli: We worked within Apexon for probably more than four years at this point been quite a long time. We had a team at place when I joined so a number of the same team members too it’s been a great journey. We’ve worked on mobile development, mobile testing. I’ve enjoyed being able to bring in some of your domain experts. I remember bringing one of your automated test experts on to our sites for a number of weeks to support us. As we’re looking at the strategy of our automated testing leveraging your router community in addition to our core dedicated team has been really helpful.

Probably, the next thing I’d point to is co-located teams. For a while we had our mobile development and mobile testing in separate locations using separate groups, separate teams bringing those together on your back office as a single co-located development and testing team. It’s been really beneficial for our mobile program and as well as seeding the IOS and Android team together. There’s a lot of sharing between the platforms and a lot of sharing between dev and test and that’s really beneficial for us.

Avery: Ed, it’s been a shorter relationship so far but I hopefully at least will be as long as the one with Proteus. What are your thought about what’s been distinctively helpful about Apexon as you’ve been working on your automation, testing and development?

Ed: Yes. It’s been an interesting relationship and it has grown in different areas as you’re aware. Apexon came on board doing primarily automation testing for LifeScan over a year ago and since we’ve challenged them to help us out with doing some of the manual test execution work and actually doing development work as well. They stepped into a bigger role in doing the mobile app development taking over for another contract organization there are two different contract organizations that we’ve had.

Also helping out a little bit now on the web side on the development area. As I mentioned before I think having expertise with a bunch of different levels is very helpful, direct J&J employees or LifeScan employees that work for me that are able to oversee and manage the processes and approve the work that’s being done and recommend the tool changes and tweaks.

We’ve worked with Perfecto a lot as well. Eli mentioned co-located teams and while we tend to use multiple test vendors on different programs I think just aligning, having Apexon doing the development and testing work, even if I think they’re in two different locations. The dev teams are in Ahmedabad and the testers are in Pune so they’re not necessarily co-located but just being within the same organization has improved that feedback loop a little bit I think into some of the past few point releases in the automation test side that’s been going on that’s been helpful.

Avery: Good, then what we’ll do now is the open it up if people got a question, there’s some more stuff where we can cover here. I also want to give people a chance to raise up any questions. Get your questions submitted, I want to start getting into that. In the case of Proteus and J&J you guys have both been — the regular term I’m going to circle back on the regulatory because I know there are a lot of regulated industries that really struggle with the digital because of this being able to agile and scrums and things are being changed under your feet by Apple or by Android, and yet you’ve got the expectations of the industry so how do you start to deal with that? What would be your advice to other people dealing in regulated industries especially healthcare but also, I think this applies to finance and other areas?

What are the things that you do that keep regulators, I’m going to say satisfied but I’m not sure you ever completely satisfy a regulator. I don’t know, so what advice you’d give to others in a similar situation?

Eli: If I can start Avery, one thing we’ve tried to do very carefully as we’ve gone down this agile journey is ensure that the end artifacts are documents that the agency is familiar with. I’ll give you an example of that, in agile there’s a concept of incremental development whether it’s incremental. Can you hear me okay?

Avery: Absolutely, yes.

Eli: Great. That concept of incremental, in addition to incrementally adding features, core agile concepts we also incrementally create our artifacts whether it be a requirement specification, a design spec, a test protocol, and in the end though we package it up in forms that are familiar to FDA. They can look at those and see them like a traditional SRS. They don’t know it was created incrementally week by week, sprint by sprint.

They see it as a whole document whether again the test reports, the protocols. That’s really important for them coming in and seeing something that’s familiar. When things are unfamiliar we found that that causes digging, that causes confusion and so as much as possible the short version of my advice is align with their expectations, don’t expect them to align with yours, your way of working.

Avery: I think that’s a watchword for all digital transformation is transform on the inside but make it look the same on the outside, don’t complicate things.

Eli: In time, they might jump on the agile bandwagon and there’s some hints that they are simply warming up to that with folks on FDA participating in various standards development or for guidance development around that. It’s really not something that we can force. We have to meet them where they are whether it’s the regulators or customers.

Avery: Ed, by the way when I was there last week, your comment that the end product is not a test; the end product is the test report. That seems in concert or aligned with what Eli was just talking about.

Ed: Yes, I think beginning with the ends in mind is what you need to do. It’s about based on some report. There was a formal test that we run and document electronically for submission to the agency are probably just a fraction of what we run throughout sprint development and our development efforts. Picking out the right set of evidence because it’s no longer 10, 15, 20 years ago, you would just like to throw a bunch of phone books at the FDA. The bigger, the bulkier, the data that you threw at them, the better it looked that you did all this testing and all this work.

What we try to do is make sure we’re testing the right thing. Even though you run these automated tests over and over again, you might not want to submit all of them. Certainly, not every execution of it, just the one evidence on the final build, on your release candidate build that you are putting out. We’re fortunate again, having part of these larger corporate resources that the quality and regulatory people that we’re engaged with have lots of experience and have lots of colleagues throughout the other industries that they can rely on.

But yet they understand what’s specific to our application and our level. We get good console on what’s required and why we need to certain things and why we don’t need to do other things, leaning it out.

Avery: How much when you are dealing with the automation, how much does automation help? In other words that you can show to a regulator, “This wasn’t just a test that I wrote down and someone followed some instructions. It’s automated, repeatable, documented set of scripts that are interacting.” Does that help or is it they don’t care; they just want to see the end result?

Ed: I don’t know that they differentiate between what’s a manual test and an automated test. I think that would fall back on. Where I think the automated test helps is providing quick feedback on a build verification. After a developer changes a couple of lines of code, he can run a full battery of some standard workflows that our customers would use and know if something broke somewhere in the existing feature and function and get that feedback back to him.

In terms of submitting to an agency, I think maybe they might have more peace of mind knowing that this test may have been run dozens of times throughout the development process. I’m not sure that they really differentiate much between automated versus manual. That’s probably a better question for somebody in a regulatory group.

Avery: That fair, I was curious Ed because in some cases, in finance they do care because it’s now something that is provable. Eli, what’s been your experience in dealing with this because this is one of the challenges of people look at innovation? They think, “Oh, I can’t innovate because of this regulatory environment,” yet you are able to transcend that issue or finesse it, I’m not sure what the right term is.

Eli: I think one big thing it’s helped us with is the packaging and the story. What I mean by that is in the old days of doing manual testing, it would often be selected regression runs as you get towards the end of a release. For example, if you have half a dozen different release candidates where each candidate has fixed a bug or two, your testing in the manual days would often span the multiple release candidates and you’d have to tell a story of why it was okay to run a partial set of tests on each subsequent release.

With a fully automated environment, you could run all the tests on every release and not take a hit in time. We had an example of that this week. I mentioned we got a release out yesterday. It had three release candidates. We thought we had the perfect release on release one and then here we are at release two and three because there were a couple of high priority bugs. We were able to run the full test suite on that third release candidate and it just makes the storytelling easier. “Here’s a release, here’s all the testing,” and we don’t need to then explain why a subset was run on each of those candidate releases. It just makes for a smoother conversation and I guess that’s on top of the story of the more testing, more often as we go.

Avery: How much of really the rapid feedback with automated tests allow really rapid feedback to developer or how much does that helped to developer productivity where they get feedback where they’re still thinking about the code as opposed to sometimes they’ve started working on something else now they have to jump back on do it?

Eli: It’s been big. It’s allowed us to think about addressing defects as we go, working them into the sprint rather than in the old not too distant past old days, we would think about feature development and then a fairly long hardening period of time. That’s been important for us to again, shrink that time from code freeze to release. We don’t carry a big backlog of bugs there at the end of our feature development.

Avery: Hey, what’s — at J&J, there’s more experience with automation helping both the regulatory conversation as well the developer productivity?

Ed: I think as I said before, it gives our regulatory affairs group more confidence as we go through the individual sprints and qualifying that the feedback is quick and immediate, development teams as they go through the sprint cycles. Just like Eli said, it reduces your testis sprint or hardening sprint at the end of the development cycle because there you are just dealing with last minute cleanup and things. Yes, automation is an integral part of it and can help build the quality in throughout, shorten up the testing at the end and also expedite those results and give you more confidence in going forward with the application.

Avery: This has been really good. Can I open it to either of you? There are questions you’d ask each other or what advice would you give to others in your similar roles that are looking at these? Because I think one interesting challenge with digital transformation is also when you start seeing competitors you never saw before. Sometimes, you are forced to change before the rest of your organization is ready. What advice would you give to people about how to deal with this?

It’s a complicated environment to deal with. Ed, what advice would you give to others that are doing this? Where should they start? What’s the biggest obstacle they need to fear? What’s the biggest benefit that they should look forward to getting?

Ed: I think from, again, I’m a latecomer to LifeScan. I missed their whole diabetes franchise that started 12, 15 years ago. I think they are only realizing in the last year or two they need to put more emphasis on the digital side. Maybe providing leadership with the knowledge that they need or selling it to them and providing examples. Whether it’s a competitors’ announcement or a competitor product is coming out or something.

Looking at better ways to enhance the release timing and release quality for digital products. There’s a lot of tools out there that are available and finding the one that’s best for you and implementing it the smoothest, best way is obviously key.

Avery: Eli, what would you say is the biggest — the things you’d give someone advice if they’re dealing with — One of the ones I clearly took away is meeting people where they are and trying to present a revolution in a very I’d say palatable easy to absorb way is one key insight. What other ones would you say?

Eli: I think another one is we’re in a fast-moving regulatory environment, I’d say relatively fast. We saw the 21st Century Cures Act come through at the end of Obama’s term. I believe that was December when that came through of last year. We’re starting to see the impact of that. Ed mentioned the PreCert program earlier allowing companies to do fast track software approvals. We’re seeing other changes at FDA so it’s a moving target. We need to keep on top of that.

One of our biggest competitors might be someone in their garage writing an app that satisfies a patient, satisfies healthcare systems in a much cheaper way. We think there’s value in that feedback loop, that’s real, a physical ingestion event. Someone might figure how to do that in a purely digital fashion in the sense of purely software rather than requiring the physical components of the wearable-sensor, the adjustable sensor.

It’s possible that app wouldn’t be regulated and something we talk about often is what might be out there on the AppStore that the developers don’t worry about FDA. They don’t worry about quality systems, they don’t particularly care about automation and testing and they can just do that cheaply and hopefully that’s not a threat anytime soon. I think keeping an eye out for what’s happening in that blurry space. It’s becoming increasingly blurry between medical devices and consumer products especially with the changes at the FDA.

That’s good news for medical device companies that have been spending a lot of extra effort on products that are low-risk profile. We got to keep on top of it and I hate coming back to people all the time but having good regulatory people in-house that are tapped into the changes in regulatory agencies it’s really important.

Avery: Thank you all very much for your time you guys you’re doing some really — I’m really proud of the work that we’ve done with both of you. We’re proud of being involved with the products you guys are creating because they’re really changing how people are getting healthcare delivered. Thank you for your time and thank you for the insight about how to actually take all this technology to market through a complicated regulatory environment to a set of customers that and partners that are struggling with how they absorb and adopt all this themselves. Ed, Eli any last comments? Well, thank you all.

Eli: Just want to say thanks Avery, I appreciate the opportunity.

Ed: All right, yes same here thanks for your support and helping us through the longest journey and good luck to those on the line that are considering taking the leap into automation or any anything in that regard with their apps that they’re working on them.

Sarah: great.

Avery: Thank you.

Sarah: Thanks everyone, Thank You Avery, Eli, and Ed I believe that that’s all the questions were able to answer this time. I just want to thank everyone for joining us today, you’ll receive an email the next 24 hours with a slide president — excuse me- of this video presentation and just this whole link to the webcast replay. For more information just please visit the link apexon.com/getstarted. Thanks again and have a great day.