Month: August 2019

Future of testing Artificial Intelligence and Machine Learning.

SaiAudityaLeave a comment

As the world progresses, we see automation taking place of human activities. All sectors are hugely disrupted by the AI and ML.

Now the big question comes, AI will take over monotonous and redundant processes where brain power requirement is lower.

The question is how software quality sector is getting ready for it?

How do big companies make use of AI to sustain for the future disruptions in the technologies?

Let’s see the glimpse of possibilities and current implementations in this section.

Artificial Intelligence, is mostly known as learning process by machine using predefined algorithms called supervised learning, the other part of it is unsupervised learning where AI engine deduces predictions by itself by recognizing the patterns example of it is edge detection in photos.

Some of these algos are inspired by the fact that how our brain works. There are several researchers out there understanding the workings of brain to simulate the same in these algorithms.

Some of the AI runs on pure statistics and might not require any learning engine behind.

Let’s coming back to original intent of the content, lets breakdown the areas of testing and see how AI is coming into all the areas.

Tests Preparation

In the quality industry identifying the proper tests for the application or business process is quite tedious process, in this context there are some interesting AI implementations.

Mining of tests – This is carried out by intelligent Natural Language Processing (NLP) algorithms

Test Selection – By analyzing the failure logs from the application, one can design learning engine to select the specified tests for the specified failures. It’s again a NLP algorithm

Test Duplicates Removal – We can remove the duplicate tests by mining the tests and similar tests

Test Creation – This is still ongoing research area where AI engine can automatically derive the test based on the user actions, like record and play back feature in several automation engines

Test Execution

Current test execution process is generally triggered by devops tools such as Jenkins or TFS. What can we do additionally? Can we use AI?

Execution – We can use speech input to trigger the test with Speech to Text and Text to Speech converters, these engines are basically developed by big companies and lot of AI research goes into it now. Amazon Alexa, Apple siri are just tip of iceberg.

ChatBots – Ask Chabot which tests you want to execute and it will pick up that for you, you can choose, group the test you want and star the execution.

Execution Results – There are few interesting use cases when comes to test execution, let’s take for example we have ran suite of several hundred tests and it generated a bulky report, then its time for manual verification of those results, if they are because of the script or because of UI changes, or because of the actual application issues. Here, first thing is to develop robust scripts, but still there is lot of scope to create an engine which will be first supervised to learn and understand the different types of failures and then predicts the cause of the failures. This is not yet implemented case, but when we think about it, there is lot of scope here to tune the engine to classify the failures. This will become classification problem in terms of the AI.

Failed Method to Test Selection: There is possibility to identify the failed method or the type of the error with above mentioned approach, and going further deep we can very well correlate them with what tests to execute for what type of failures, what methods are failing and what are the expected tests to run when the application gets new fixes.

Performance Testing

Machine Performance Analysis – This is the actual area where AI shines. Mostly the data will be plotted in graphs in all tools like LR and Jmeter. But we don’t know the anomalies and insights of the data. We can detect changes in the Disk performance (iops), we can detect anomalies in the CPU utilizations, and one can correlate them with actual transactions happening at that time. All will be done with supervised learning or unsupervised learning. We can implement different algos and see how they are performing.

Transaction Performance – This is good use case similar to above. We can collect the metrics of all transactions and apply different alogs to detect anomalies, peaks in the data. By which we can get to few conclusions. This is the area I have done prototype which I will be sharing in next blog.

So, all in all, there are several use cases which AI can improve the way we work. But nevertheless, it will not replace the ability to create something new something very creative like identifying security bug etc, may be anytime soon.

Please write down below in comments if you think anything can be done with AI in testing or we have already done we can discuss about it in detail.

 

 

 

 

 

Protractor setup and example of non angular app simple test – Part 2

SaiAudityaLeave a comment

In the Part 1 we have seen how to setup the protractor, In this section lets see how to write test spec file and configure the formatted report.

googleTest spec file has below content.

browser.ignoreSynchronization=true;
//The above line is specifically written for non angular applications such as google
browser.driver.sleep(5000);
//Describe the test and expected result in it block
describe('google homepage', function() {
it('should open the page', function() {
var waitEC = protractor.ExpectedConditions;
browser.driver.get('https://google.com');
browser.driver.sleep(5000);
browser.driver.findElement(by.name('q')).sendKeys('Testing Example for Protractor non angular');
browser.driver.sleep(1000);
expect('Testing Example for Protractor non angular').toEqual('Testing Example for Protractor non angular');
});
})
describe('click on search', function() {
it('should click on search button', function() {
browser.driver.findElement(by.name('btnK')).click()
});
});

The above code is for the protractor spec file to open the google website and search for specific keywork and click on google search button. In the next section will see the formatting part.

As we know, we have different types of formats available for the protractor, among those below is widely used as it is fortmatted console and does not require additional css or etc files to view the results. All results are shown on the console logs it self.

we need to first install the node package for this.  More info jasmine formatter

npm install jasmine-spec-reporter

Once we install the package, it will download necessary module under protractor location.

Once downloaded, we just need to update our conf file with the downloaded formatter.

The sample conf file also given in the above site. Following is the sample conf file used for our googleTest spec.

let SpecReporter = require('jasmine-spec-reporter').SpecReporter;

exports.config = {
framework: 'jasmine2',
jasmineNodeOpts: {
showColors: true,
silent: true,
defaultTimeoutInterval: 360000,
print: function () {
}
},
specs: [
'googleTest.js'
],
capabilities: {
browserName: 'chrome',
'chromeOptions': {
args: ['--test-type']
}
},
logLevel: 'WARN',
onPrepare: function () {
jasmine.getEnv().addReporter(new SpecReporter({
spec: {
displayStacktrace: true
},
summary: {
displayDuration: false
}
}));
}
};

Once we have updated editing the conf file. you will be able to see report in nice colored format shown below.

Result

Please write to me or down below comments section if you are facing any advanced issues in protractor.

Thanks for reading..!!!

Protractor setup and example of non angular app simple test – Part 1

SaiAudityaLeave a comment

Now a days we see lot of angular JS applications being built.

It is said that at least 10% of the latest developing are based on angular js.

In this section we see how to configure and run a simple protractor test on angular and non angular app

Its very straight forward, mostly thats the reason why its largely adapted.

Step 1 :

Install Node.js, current version of node js when writing this blog is v10.16.2

you can download node js from  Node Js download

select the windows MSI and install it. While installing it select add node js to path.

This option enable us to directly access node commands from cmd without navigating to node directory.

Node JS is like collection of several java script libraries. They are called as node modules

The installation also installs the node package manager which is required to install different node modules.

Once you have installed it  go to command prompt (search-> run – >cmd)

then enter following command

Node -v

It should show the current installed version, if its not there is some issue with the installation.

Step2:

Now we have node js setup and ready

Open cmd and try to install the protractor, install protractor globally (all users on the machine, if not remove -g)

npm install protractor -g

More information on this site https://www.protractortest.org/#/

Usually protractor sets up the web driver manager. But if it’s not then you need to install the webdriver manager as well

webdriver-manager update

Step3:

Now we have to start our webdriver-manager to kick of the example test

Webdriver manager will look for webdriver connections that are being initiated by protractor.

webdriver-manager start

There is one sample test that comes with protractor we can use it and create couple of more tests.

But, if we want to run on google we might want to make some changes.

At the end of start of webdriver you should see command prompt like below.

Capture

Step4:

Once you started the webdriver manager, open another command prompt and run a angular example test

In the node js installation will happen typically in appdata/roaming folder go to following path, you should see following

Example path :

C:\Users\{user_name}\AppData\Roaming\npm\node_modules\protractor\example

You should see the following, googleTest is something I have added, you don’t see it by default.

scriptlocation

To run a test we need to use following command

Protractor C:\Users\{user_name}\AppData\Roaming\npm\node_modules\protractor\example\conf.js

What does this mean ?

Protractor reads the conf file and executes the specification file mentioned in the conf.js

Protractor uses something called Jasmin framework by default. Its called as “Describe It” framework.

Test start with Describe() – > then It() , which means action and its expected outcome.

All the code is written in java script, we can use type script too.

Once we start the example test it will perform operations on node js site and gives us the following result.

The result displayed below is formatted, I will mention how to configure different types of reports in next section.

Result

In the next part will discuss about following

  • Custom report as shown above.
  • conf.js sections and how to edit.
  • googleTest.js -> non angular application example.