This website uses cookies to offer you the best experience online. By continuing to use our website, you agree to the use of cookies. If you would like to know more about cookies and how to manage them please view our Privacy Policy & Cookies page.
A joint presentation with Tricentis
Josh: Hello and welcome to our webinar, Salesforce intelligent continuous testing for the digital enterprise. My name is Josh and I will be your host today. Today’s presenters are Manish Mathuria, co-founder and CEO of Apexon, and Sai Niveda, product manager at Tricentis. Manish is an experienced technology consultant and business leader with a passion for building winning teams. In his position as chief operations officer, he is responsible for global delivery operations and innovation. He has served as a senior technology consultant across quality engineering, software development, cloud, SAS and mobility for Fortune 500 firms and Silicon Valley startups. Sai is a product manager at Tricentis and has been in the testing industry for over 11 years. She has extensive experience in automation with a proven track record and implementing strategic testing solutions that help businesses overcome testing and process automation roadblocks. Her focus at Tricentis involves defining product strategy and designing innovative solutions to automate SAS applications such as Salesforce, ServiceNow, Microsoft Dynamics, 365, et cetera. Say hello you guys.
Manish Mathuria: Hello.
Sai Niveda:: Hi.
Josh: Thank you. Before we begin, let me review some housekeeping items. First, this webinar is being recorded and will be distributed via email to you allowing you to share it with your internal teams or watch it again later. Second, your line is currently muted and will stay that way throughout the course of this webinar. Third, please feel free to submit any questions during the call by utilizing the chat function at the bottom of your screen. We will answer all questions toward the end of the presentation and we will do our best to keep this webinar to the 45 minute to one hour time allotment. At this time, I will go ahead and turn over the presentation to Manish.
Manish Mathuria: Thank you, Josh, and thank you, Sai, for joining me on this webinar. As the title suggests, today we’ll be talking about intelligent continuous testing, particularly around Salesforce. So as part of the agenda, we will walk through why continuous testing and why best automation is so important for Salesforce. What are the pressing demands in the marketplace, some of the trends with which Salesforce has been evolving, particularly the challenges around Classic to Lightning transformation and how best to describe all of this without a case study. So we’ll talk about a specific case study that Apexon has been through among many of our customers. And finally, we will have Sai demonstrate how automation can be done with Tosca easily, intelligently, and leveraging the latest AI technologies. So let’s get to it.
Manish Mathuria:: A little bit about Apexon. Apexon is a digital engineering outsourcing company headquartered in Silicon Valley, California with offices in New York, UK, and delivery centers out of India. We are about 1200 engineers strong and we enable our customers implement state-of-the-art next generation digital solutions for their companies. I’ll ask Sai to explain about a Tricentis now.
Sai Niveda:: Thank you Manish. Tricentis is the leader in continuous testing and automation. We have a presence worldwide and are at a lot of the top companies globally as you can see on the slide. We work very closely with partners including Apexon. One of the things that we help customers with is in the continuous testing space, basically supporting and accelerating their digital transformation journeys. Over to you, Manish.
Manish Mathuria:: Thank you Sai. All right. So let’s look at why test automation is so important with Salesforce. Test automation is a critical component in the end to end DevOps processes. As we all know, DevOps is another name for optimizing and automating your entire software development life cycle around Salesforce. So this means the act of development, building, deployment and testing and ensuring quality through this entire process. Lots of studies, external studies, as well as our internal benchmarks while working with our customers have taught us that there is a significant ROI in implementing DevOps processes around Salesforce.
Manish Mathuria:: Some of the numbers are explained here. For example, our customers have been able to achieve 46 times more deployments in the same time period using automated testing and automated DevOps processes because it ensures that there is not that much human intervention in the end to end deployment process. There is less chance of failures. If you fail, there is a faster recovery time, again, significantly higher and you save as much as 40% reduction in the cycle times and the deployment times due to automated testing. All of these characteristics provide a significant ROI when you look at automating, testing and other DevOps processes as part of your Salesforce software development life cycle
Sai Niveda: And Manish, this is also something that customers generally tend to struggle with implementing right.
Manish Mathuria: That is absolutely correct, Sai. In our experience, particularly around Salesforce, the implementers understand Salesforce very well and they understand the process of implementing Salesforce and the business demands around it quite well. But when it actually comes to implementing DevOps practices, test automation, the teams struggle to do it correctly. So even though there is so much ROI and there’s so much value, most customers struggle to actually do right amount of test automation around it and hence get impacted in their release cycles.
Sai Niveda: Yeah.
Manish Mathuria: Again, this slide talks a little bit further about the specific challenges teams face when taking Salesforce through the deployment cycles. Like I explained, DevOps cycles, automated DevOps cycles implement, achieve into an automation all the way through development, doing the build deployment to various environments and testing in that process. This achieves significantly high quality time to market, reduction in failure deployments and significantly reduces meantime for recovery. We have seen that on an average, this increases cycle time at least about 50% in your end to end deployment cycles. Salesforce is-
Sai Niveda: Sorry Manish, [inaudible 00:17:40].
Manish Mathuria:: I was just going to add, Salesforce is somewhat unique in the way DevOps processes get implemented. And that is one of the reasons that you were mentioning why it’s so challenging to deploy DevOps practices around Salesforce. A, it is cloud. B, it is not your traditional custom software development environment. It’s an environment in which there is a lot of customization and there’s a lot of prebuilt package software that needs to be customized. Hence, it is extremely complicated to create an environment where a lot of this automation can be achieved via your traditional check in and get repositories and so on and so forth.
Sai Niveda: The other thing also is, given how easily customizable the platform is, the frequency of changes tends to be quite faster compared to the other applications in the space. So you can expect higher turnaround with newer features coming in and deployments and things like that. So the frequency is something people need to watch out for.
Manish Mathuria: Exactly. And in my experience, there are many causes of this high-frequency. It is not just Salesforce deploying a new version on you, but your business processes change. Salesforce would often be in, not an isolated world, but it would be amid various other enterprise applications across which these business processes are automated built. So changing any of these applications could cause you to test and release and certify your Salesforce implementation. All of these things actually combine together to actually make it a very sensitive testing process. Would you agree?
Sai Niveda: Yes, definitely.
Manish Mathuria: Right. Let’s move on. Now, when we talk to our customers, we often get asked, should we worry about automated testing? And is it for us or can we do without it? Over the period while talking to various customers, we created these three categories of companies who are implementing Salesforce. For lack of better terminology, let’s just call them small, medium and large enterprise. Here are some of the characteristics with which we recognize what is small, what is medium and what is large. I think large and medium are somewhat more interesting here. So you would find that in a medium to large enterprise, there would be definitely a fair bit of configuration of Salesforce. But in addition, there would be a lot of custom code. And in addition, there may be integration like I was explaining, with various other platforms across which various business processes are weaving in and out. Due to which there may be a high deployment frequency, the team sizes by what you are building around it could be complex and large number of developers and hence the flux that creates between these developers through these testing and DevOps processes is very complicated.
Manish Mathuria: Therefore, the time it takes for one to release a change to the production often without automation can take from weeks to several weeks and it is often a big impediment to progress of business. So we find that if you are a medium to a large enterprise, certainly automation can go a long way in achieving that highest cycle time as much as reducing your cycle time by almost 50%. Any comments, Sai?
Sai Niveda: One thing around the medium enterprises, while it’s mostly perceived that we can handle these things manually and stuff, that’s the gentle feel from some medium enterprises at least that it can be handled the way they’re doing it currently, testing things manually and stuff like that. But actually over time, the complexities of just the configuration tend to get pretty high and that’s when you’re basically in that hyper care mode where actual end users are reporting issues back to you when you’re trying to perform these hot fixes in production. So the necessity to actually have that continuous testing in the space, especially from a medium enterprise onwards is pretty important.
Manish Mathuria: Okay. Moving on. So let’s look at this from a case study perspective. I think it’s best to explain the challenges that we typically run into across these medium to large enterprise customers via an example case study that we recently went through. I don’t think this picture would be very complicated if you call yourself a medium to large enterprise. This particular customer was trying to automate and enable their sales marketing, service delivery and customer support business use cases through implementing Salesforce and a myriad of other products which form that entire enterprise stack. They are a medical device company, and therefore successful deployment and execution of these processes is critically important to their care provider customers and their patients. So as you can see, there are various products that come together to automate these processes that I talked about and there is an integration technology underneath. In their case, it was Dell Boomi, but we have seen MuleSoft, we have seen several others that get deployed in such scenarios. Right?
Sai Niveda: Yeah.
Manish Mathuria: A little bit about the functional coverage they were trying to provide. Like I said, this spans across various business functions. I won’t go into the specific use cases, but needless to say that this huge range of functions that they are looking to automate around, particularly around Salesforce is a multi year, many developers a project where from release to, they realize that putting something to production without achieving automation and streamlining their processes, DevOps processes was a incredibly complex tasks.
Sai Niveda: The other thing also, Manish, to add to this is depending on how the automation is set up, customers also tend to find that they’re always playing catch up. So this is a very complex landscape. But actually keeping up with all of this from an automation perspective also proves to be quite challenging.
Manish Mathuria: That is true. In the remaining part of the case study, I’ll touch upon that topic and, and talk specifically why it is so complex. This is one of the moderately complex use cases that this customer had. As you can see, one of the complexity here is that one business use case actually spans across various enterprise systems. And each of these systems have to be in the right state for that particular use case to successfully execute. This use case had in excess of 40 different test scenarios, and each of the test scenarios exploited a specific state or a specific nature of a work order or whether the parts were internally manufactured or externally manufactured and various other combinations of challenges. To touch up on why automation is complicated, as you can see, this is not just Salesforce. So if you are looking at identifying which product you’re going to use, which test tool are you going to use in your testing stack, you’re to keep in mind that it is not just Salesforce you’re automating. The product that you pick better support testing across these enterprise systems.
Sai Niveda: The other thing is also, what we’re basically trying to do here is watch the journey of a piece of data from say, a Salesforce across all of the systems that you have within the enterprise landscape. That becomes important to track what is happening at any given stage of that data’s journey.
Manish Mathuria: Right. And speaking of data, that is yet another very interesting and very complex part in an end to end scenario like this. What we found before we got in was that before the customer could even think or talk about testing, they had to make sure that each of these systems with respect to data, were in the right state. So along with actually doing automation or testing for them, we had to actually cater to this data requirement. And again, the test tool that we chose, which we will talk about in a little bit, having an internal test data management system came significantly in handy. While we are on the subject of this use case, I also want to highlight another very important characteristic here.
Manish Mathuria: Often we found that these individual systems were not in the right state for us to test and if we waited for each one of them to be exactly aligned with one on of that so that we could test this end to end system, we would be losing very important time that we could achieve while we could be testing one system. We used a concept called service virtualization, which essentially allowed us to simulate or mock another system while we were testing one particular system. To give an example, when we assigned a work order to ServiceNow, that particular API call that actually assigned the work harder to ServiceNow, we did not actually use ServiceNow until we really reached the end the life cycle of testing it. On our all three pre-staging environment tests, we were able to simulate all of this by leveraging a concept called service virtualization.
Sai Niveda: Yeah. So the importance of testing in isolation and then testing together in your upper stages before the entire stack goes onto prod.
Manish Mathuria: Exactly, exactly. So looking at some specific challenges around testing and release cycles, we already talked about the fact that two week release cycles are a norm these days. If you’re doing anything beyond that, you are already significantly behind the market. Our advanced customers or mature customers actually can release to production multiple times a week in a fairly automated process. The release challenges are significant because this is a continuous and iterative enhancement and development environment. The test cycles, speaking of tests, particular test cycles and testing landscape.
Manish Mathuria: Without automation, the regression testing can take days. You will find that there is a lot of a hidden or tribal knowledge across various teams in an organization. For example, certain use cases only business would know, or businesses capable of executing end to end. So when you actually talk about end to end testing, you have to get on these giant conference calls and meetings, web meetings where you’re actually doing collaborative testing, which is literally wasting tens of hours from everybody. We talked about data conditioning problems and we also talked about the need for the service virtualization and why it is important. As we move along, let’s look at specifically how we pick the particular tool. I’ll let Sai talk about Tosca and its capability around Salesforce best automation.
Sai Niveda: Thanks Manish. Let me just try and get my laptops started. With Tosca, addressing the challenges that we’ve got within the Salesforce space and the automation, basically touching upon a lot of things that Manish spoke about earlier. For the purpose of this webinar, we’ll be focusing purely on Salesforce capabilities from an automation perspective. But we do have support for over 150 technologies along with stateful data management service virtualization and things of that sort for you to be able to test in isolation. But particularly within Salesforce, we have a no code approach to automation. There’s an interesting feature around this that I’ll cover when we’re doing the demo. Essentially to handle the dynamic nature and the hidden complexities within the Salesforce ecosystem when it comes to automation.
Sai Niveda: We also have a couple of features that allows you to ensure that you get maximum coverage with the minimum number of test cases so that you can actually have focused testing activities around the releases that are coming through. Just as an added benefit, the test cases that you create with Tosca will work on classic and lightning. So you don’t need to make any changes if you’re in the journey to transitioning from Classic to Lightning.
Sai Niveda: All right. So I’ll just jump into a bit of a demo. Before we look at what’s actually happening on screen, for those of you who aren’t familiar with Tosca, just think about it as there are three things that you need to keep in mind when you are approaching automation. Regardless of whether it’s Salesforce or any other platform, you would be looking at what do you need to interact with. That’s basically your elements on screen. And that’s where a lot of people spend some time. So while I’m walking through the demo, the first part that we’re showing here is basically capturing the elements you need to interact with. Now this part, we’ve introduced a feature called the Salesforce scan. Let me key in my credentials here and while I’m doing that, let me explain what this section or the technical model section does. This is basically telling any tool what does this element that I need to interact with look like.
Sai Niveda: It’s very easy for a user to look at it visually on screen. But explaining that to the tool in terms of how you’re addressing this, say exports, stuff like that, it does get a little too technical and isolating the business in the process. What you saw earlier was me basically keying in my credentials and picking the production environment. Although it’s not realistic within your own spaces, you would be using an upper sandbox, say maybe a UAT or an SID sandbox. This essentially queries the metadata layer and fetches every single element that you can interact with within your Salesforce org. So I’m going to go ahead and click on select all and scan. Now you can see that there’s a progress bar at the bottom that’s basically [inaudible 00:34:39] your Salesforce instance.
Sai Niveda: If you think about it, there’s no two Salesforce instances that’ll look alike. So even the use case that Manish specified earlier is very specific to that customer. So you can’t reuse everything that you have in that customer space within your environment because your business processes change. So what this has essentially done is picking the elements that are most relevant to you. The login module, the navigational elements, save buttons. Let’s take a look at an account, for example. All of this is named according to what is there within the Salesforce organization. Also, roping in the administrators so they can understand what these layouts mean. Now, from a technical identification perspective, we’ve eliminated things that could change. So when Salesforce does change the UI, the labels of the elements are under your control. I would just click through all of these elements. So you can see that the only identification mechanism is an associated label. Also, what this does is brings up only if your values that you can possibly select from a picklist, making your test case building that much easier.
Sai Niveda: Okay. Now that you have the elements you need to interact with, it’s about how you put this together in a test case to create a logical process flow before you can reach your verification goals within that space. The process of assembling a test case is relatively simple in Tosca, like we said earlier, no code approach. So I would just drag and drop these sections of elements onto the test case layout, put in my values. Now in this case, I’m navigating to the contact screen. Once we navigate to the contact screen, we need to click on new. Dragging and dropping the relevant module for that, renaming the test case so that it makes sense within that context. We spoke about isolated knowledge between the business and the automation teams and the business not being able to contribute to automation, which in itself is a disservice to organizations.
Sai Niveda: Now that I’ve gone ahead and clicked on new, I will be entering some contact information on the field. Again, rename for it to make sense in the context. Start entering your data. Now in this case we’re using static data, but this is where some of the features that Manish spoke about earlier, things like that for test data management. Things like test case design to optimize your workflow. There are tons of features that we can use, but we will spare you for the purpose of this webinar. As you can see, it’s relatively simple to start creating it, picking your action modes, what your verification points are, et cetera.
Sai Niveda: Once you have your elements sorted and you’ve put your test case together, you can see that the same test case would work on Classic and Lightning. You can see Classic on the bottom left of the screen and Lightening on the top right of the screen. Keep in mind that there have been no changes made to the test case. So if you are on your journey transitioning from Classic to Lightning, your test cases are already done. Different user experiences, different UI, different technical information from a Salesforce perspective. Right. So Manish, could you walk us through a couple of your experiences with the platform?
Manish Mathuria: Yeah. Sai, it’s an important point you’re bringing up. We as a service company not just work with one product, we work with a lot of open source and several other tools in the marketplace. What we have found is that when our engineers work with Tosca, we don’t need a specialized skilled person. We have business analysts who are very well equipped to understand Salesforce and really are very comfortable with Salesforce processes. They can pick up the tool and start to automate test cases. So our ratio between our technical staff and the business of our staff changes significantly when we are working on Salesforce, particularly around Tosca. I consider that as a huge benefit because it allows us to be a lot responsive and execute much faster on our projects.
Manish Mathuria: I want to pick up here again, what we want to talk about also is around Tosca, what else is needed. So if you look at like an end to end stack, this is what we ended up implementing for the particular customer we are talking about. This in fact is a very typical stack that you will find on pretty much all our projects. So we do the actual story grooming and identification of the stories, your stories through our sprint cycles in JIRA. We bring those best cases into our test management system called Tricentis qTest. That is where we evolve these test cases and identify and map them to the user stories inside JIRA. From there we pick up these test cases and automate them through a process that is very similar to what Sai just described in Tosca.
Manish Mathuria: Here you can see that using Tosca all the way from test data aspects of it, virtualizing the services, making user interface tests, we also call them GUI tests, exploratory tests, API tests. We don’t have to use various different technology in tools. We use one tool to do all of these things from within that particular Tosca tool. [crosstalk 00:40:54].
Sai Niveda: [crosstalk 00:40:54], Manish, because the UI automation, especially within the Salesforce or the SAS world is what customers tend to struggle with the most because it does take quite a while to stabilize what you need from a desk suite perspective. So just in case the point was missed during the demo, it takes you a couple of minutes to have your basic elements of UI automation ready for Salesforce and you don’t need to deal with going through it screen by screen and understanding what to pick, how to pick it, and how do I get a stable automation suite? So it’s makes the process substantially faster.
Manish Mathuria: The process, Sai, that you showed, that was done automatically, which actually sock the entire object structure, automation object structure. The GUI widgets that Salesforce had on basis of which we will build the automation in matter of seconds is by industry standards and metrics. Our test automation engineer spends about 40% of time maintaining, remaintaining that, building that in automation code. So we can easily translate that to at least 40% saving in building up those test cases.
Manish Mathuria: All right. Some of the benefits that we would typically get because of automation, these are data points from this particular case studies, but you will find very typical, very similar outcomes if automation were done right in your Salesforce project. One of the critical metric that we track and we track for this particular customer was defect leakage to QA environments. So before UAT, where business folks actually do UAT in the QA environment, we were able to reduce defects by 70%. The regression test cycles, we reduced from two weeks to four hours. We were able to, using this test, data management, service virtualization and various other technologies, we were able to automate scenarios which otherwise would not be automatable.
Manish Mathuria: We reduced significantly the test team size because fundamentally, once the tests were automated with a small number of people, we were able to upkeep those tests. Of course, any automation requires maintenance and instead of executing a lot of tests manually and really doing this [inaudible 00:43:48] web meetings to do test execution, you can actually spend that energy in just test maintenance. So those are some of the benefits that this organization got from test automation.
Sai Niveda: One other thing also, when you do have these distributed capabilities between a platform that’s been picked, it gives you a little more flexibility to play with larger scopes. Like things that would have traditionally been descoped and there’s suddenly ways in which you can achieve that and make sure your system is operating at the highest quality.
Manish Mathuria: Absolutely. A real quick slide on what Apexon brings out of the box. It’s not just that we know how to automate Salesforce, however, we also bring a fair number of prebuilt assets that we can quickly put together. This could be test cases, this could be several components that are recreated that we can actually deploy them towards your Salesforce automation. We also bring best practices on what are the right kind of test cases to automate, what should not be automated? We have dashboards and metrics that we can quickly deploy and put together a dashboard around either your GR system or any other test management system as well. Essentially we can very quickly start a project in matter of days as opposed to sometimes it could take weeks for a team to get started with automation.
Sai Niveda: The interesting thing here, Manish, also having worked with your team quite closely, that ability to tell when you should, even within the automation space, when you should test something by the API versus UI, that’s something that a lot of people tend to struggle with. So bringing that knowledge on board has helped quite a bit. If you’re doing this, do this with the API because this makes the most sense. Keep this to the exploratory level. That’s been a pretty good experience working with you guys.
Manish Mathuria: Right. If you resort to testing everything to through the user interface, that could land your test cases brittle, very time consuming and maintenance oriented and actually lengthy to execute. So absolutely making the right calls there can make a difference of night and day in your overall test execution strategy.
Sai Niveda: Yeah.
Manish Mathuria: All right. In the way of slides, this is what we had to present. We are open to QA. Josh, do you have any questions for the panel?
Josh: Yeah, thanks guys and I appreciate your time. That was really interesting. So yes, as Manish mentioned, let’s go ahead and open it up for questions. You can submit a question at any time using the chat feature there in Zoom, it’s either at the top of your screen or the bottom depending on where you have chat located. As a reminder, we can go ahead and submit a question any time. Looks like we’re getting a few in. I got one from earlier while Sai was talking, so I want to get that one started first. So for Sai, a question came in about, how can QA team members improve their understanding of Salesforce testing requirements?
Sai Niveda: That’s actually a very interesting question, Josh or whoever asked it basically. There’s a few things that you could do to help you get started. One is upping your domain knowledge. Understanding the domain that you’re operating in and how you need to work with Salesforce and the nitty gritties of Salesforce in general. Now, Salesforce has a great platform called Trailhead. It’s a learning platform. It’s free. You can start getting used to what the domain and the terminology around that space is. Now also given what we spoke about earlier in terms of the no-code approach and following terminology within the Salesforce space, bringing the business in to see how they’re using Salesforce will help you understand how best to address the most risky areas for the business in general.
Josh: Okay. Thank you for that. Manish, we got one here for you. Someone asked, how do you determine what kinds of tests to automate in Salesforce? And also they’re asking what tests to leave manual. That’s good.
Manish Mathuria: Yeah, I think that’s a very important question. I’ve seen teams struggle with that. Particularly this is very pertinent when it comes to a packaged application. You didn’t write that application. So should you be testing what that application does or if not, then what should you be testing? It’s important to keep in mind that Salesforce is Salesforce. Over on top of that, you have built your business processes, you’ve interlaced that with your test data and you have built this interesting use cases that, like an example that I showed on earlier slide. Particularly in the case of packaged application, Salesforce not being an exception, keeping an end to end business view in mind helps as opposed to doing more unit level tests, each test a particular module, because chances are that if you’re testing a particular module that would any way be covered in an entwined test. So we focus on end to end tests. We try to prioritize them, of course with the business priority and sensitivity to the impact that it would create on whatever function it’s serving, revenue, cost, time, et cetera.
Manish Mathuria: Finally, in that process you will also discover things that are probably not very easily automatable. To give you an example in the use case I described, the process got lost inside SAP for a while and then it triggered the process to be restarted within Salesforce through an external trigger. And it was very complex too for us to automate. For certain reasons, we had to automate it, so we actually did it, but we argued a fair bit that, does it require to be automated? Can we test that particular piece manually? So it’s not a simple answer, but some of those are criteria.
Josh: Yeah. Very interesting. Thank you. Got one here, Sai, for you just came in. Someone says, my organization is planning a move from Salesforce Classic, I guess, to Lightning. What changes in testing should we prepare for [inaudible 00:51:07]?
Sai Niveda:
Manish: Just a couple of things now based on the features we showed you, your base test cases that have been built can still be reused in Lightning. Of course, provided you’ve used a tool that supports both interfaces seamlessly. In this case, Tosca does have the exact same test case having the ability to run on both modes of operation within Salesforce. Couple of things you need to keep in mind here fundamentally and functionally, there are a few differences between Classic and Lightening.
Sai Niveda: Now for example, you may need to do an extra step of navigation before you can get to a screen within Lightening, whereas you didn’t have to do that in Classic. These sorts of minor differences need to be kept in mind. So you may need to make some tweaks, but these are generally more predictable so you can preempt these changes and have them all in place. Other than that, [inaudible 00:52:04] part of your test suite is absolutely reusable across both. Knowing what features and when these features are coming out from a Salesforce administrator perspective also becomes important. So be in touch with a team that’s actually taking care of the migration because they will have a report of the things that need to be done for a successful migration within Salesforce as well as a roadmap for when these are going to be implemented. So your tests can actually be ready much before these changes come into your environments.
Josh: Okay. Great.
Manish Mathuria: One best practice, Sai, we followed is to keep, we painstakingly keep our test cases very, very modular. So they are comprised of really these small functions that we can reuse across various tests. The benefit of that is that, as you said, between Classic and Lightning, there could be that one extra step. So if you have to really create a different test in this [inaudible 00:53:12] for Classic and Lightening, right then it is just that the rest of the test is still comprised of those various fundamental modules. There’s this one small statement that allows us to actually differentiate between the two as opposed to [crosstalk 00:53:29].
Sai Niveda: That’s very important. Structurally, how test suites are maintained is very important because it shouldn’t be a matter of having to rewrite your test case. It should be a matter of either putting in a small conditional statement or a tiny little reusable step from somewhere. And the extent of which you can reuse test artifacts is super important and keeping up with the changes here.
Josh: Okay. Thank you. Another question for Sai is, someone’s asking if you guys keep up with your Salesforce seasonal releases. I know Salesforce makes occasional updates to the software, so are you guys, how fast do you update your site as well?
Sai Niveda: Just realized I should have probably covered that in the slide so far, but [inaudible 00:54:18].
Josh: That’s okay.
Sai Niveda: We lost Salesforce release agnostic because we don’t depend too much on the technical layers of each of these. We have our engine that takes care of that for you. In the event that there is a drastic change that Salesforce has introduced, especially the UI layer, the maximum you would have to do is maybe install a tiny little patch but you would not have to make any changes to your test suite. So we do keep up with Salesforce, the three releases Salesforce has a year.
Josh: Yeah, great. Excellent. Okay. I have another one here for Manish. How frequently should the automated tests be run and in what environment?
Manish Mathuria: Right. What we do is we tag test cases with the tags. So we basically tag a test case to be run as a build verification test. Or to be an end to end test or maybe a priority one test and so on and so forth. We can arbitrarily mix and match these tests to form these suites in Tosca that could be run appropriately for the right organization. So the right answer is that you would potentially be running different tests in different environments. After you do a build, your continuous testing system, your system will probably run a suite. That would be very different after you’ve done a deployment maybe in a QA environment or in a staging environment. So this is done for purpose. The test selection is done for purpose to which you are actually doing the test, if that makes sense.
Josh: Yeah. Okay, great. I do still have a bunch more questions but unfortunately, we are short on time. I want to make sure we get to them. For those of you that have submitted your questions, we will respond to them via email. So please, know that we have your questions and we will get back to you shortly in the next couple of days or so. But I want to go ahead and thank our presenters, you guys did a great job. Thank you for participating. And many thanks to Manish and Sai. Thanks everyone for joining today.
Josh: You will receive an email in the next 24 hours with the slide presentation and the link to the webcast replay. If you have any questions, please feel free to contact us at info@apexon.com, or you can reach one of the presenters. You see their emails here on the screen. Be sure to check out and subscribe to DTV, which is one of Apexon’s digital transformation channels that we have that brings industry experts, including Tricentis and others to market. Thank you all and enjoy the rest of your day. Thank you and have a wonderful day.
