Test Automation - How to begin

I hear questions/screams are floating around, "I want to learn automation" or "How should I start learning test automation"! While these are very important questions but sometimes I read some utterly painful replies to this newbies. I feel horrified when I see replies demonstrating how to learn selenium in response to that. I have also seen people describing test automation pyramids, ideas to learn testing first which are required, but aren't we missing some basics here?

Lets ask Google!


I asked Google and Google responded with lots of useful information (as it always does!). After browsing few articles I realized that community of questioners are mainly divided into two level: 
  • Newbies to test automation but experienced manual testing professionals and 
  • Newbies to over all testing and want to begin with test automation
For both of above mentioned professionals, it's a valid question but need to learn some basics to begin with. In this article I'll present few items which I believe are most important to start learning test automation (or any automation which requires programming 😊 ).


Selenium is not automation

Yes, please keep it in mind. Selenium has nothing to do with test automation learning. When I interviewed people, they often mention that they are learning selenium. Then immediately I reply with question What actually you are learning in selenium? Which programming language you are comfortable with? Then the mumbling part begins. I see candidates are actually learning tools instead of learning programming basics.


So, what's suggested?

I suggest to step back one more step and start with asking yourself the questions:

  •  Am I the right person for test automation?
  • Am I really passionate about programming and designing?
  • Do I have enough interest on programming or I am more comfortable with manual testing?
Trust me these are very important question. I have worked with team where 20% of the team were actually interested and adopted quickly with automated testing. But rest of 80%, unfortunately, were happy with manual testing.


Change Your Mindset

Please don't be reluctant about this. If you are coming from manual testing then it's even more important. You need to start thinking like programmers/designers. Whenever you are testing a feature, think till which extend this feature could be automated. Then think, if you want to automate it what are the available options you have. 
Also you need to think if you are gonna design the testing framework from scratch, If you are certain about designing a framework then prepare steps for framework first. If you see that you are able think in this way and able to answer yoursel then definitely you have moved the first stone!


Work on your programming skills

Again, Please don't try to learn Selenium or Watir or WatIn or Sahi or Robot Framework. These are nothing but few frameworks to ease our automation efforts. But underneath you need to work with a programming language (Java or Python or whatever!) to utilize these properly, and to do so there is no other alternative than improving the programming skill.
Another important are to work is logical development. Because by the course of time, you'll see that sometimes you may need to establish some test code which requires to work on performance or time complexity. So learn basics of data structure and practice programming. Sometimes what I do, I go through programming lessons of Codility and try to solve. I am sure that if you search, you'll find thousands of alternatives to that.


Start exploring tools/frameworks

This can be done in parallel while you are preparing yourself for test automation. Instead of beginning with these frameworks (selenium or others), start exploring them bit later. Learn pros and cons of frameworks, asses based on your need and select one.


Pick the simplest to begin

Identify the simplest task to begin with, may be automating a search in google can be easiest place to start. Then finalize your framework for that simple test, when you are confident enough, then start enhancing your framework for other projects. 
Or if you see that your company is asking for test automation, then can begin with company project straightway, but again, start with the simplest operation of the project.

Happy testing, happy coding :)

Python packages - I have worked with so far

I am acquainted with Python for almost 2 years now. First I worked with around 2013 if I am not mistaken. It was the client requirement that time to use Python. So we started from zero, learned it and delivered the testing framework to the client. Client's response was satisfactory. But after that there was pause until someone else suggested me to try Python for few of my automation.

That "someone" was my development manager here, it was brief timeline that I spend with him before he moved, but within that brief time frame he changed my perspective towards to the automation. From that time I started automating things in all possible way. It's not only UI automation. I have developed different frameworks for my current organization.

I built one REST API Testing framework (actually cloned from https://github.com/chitamoor/Rester). I build another framework which I named PyRabbit, to help us to test each of our microservices. There is another recently I finished development is called "LocalDeployer". Purpose of this project is to allow download artifacts from our Bamboo CI and deploy microservices with a single command.

There are few other projects I have been working with which are out context of this article :).  But, what I want to share is the packages I have used so far while I was/am working with so many versatile projects. The packages I have used so far are as follows:

Requests

Title of the doc is "Requests: HTTP for humans", which is kind of correct. If you ever want to deal with http communication including POST, GET etc and different types of authentication, then should check out this package. Requests API is easy to use. Dig down further details at http://docs.python-requests.org/en/master/.

Note: if you ever wonder how to do OAuth2 authentication using requests, then please checkout https://requests-oauthlib.readthedocs.io/en/latest/.

PIKA

It's a python implementation for AMQP (Advance Message Queuing Protocol). From the doc and I quote:
Pika is a pure-Python implementation of the AMQP 0-9-1 protocol that tries to stay fairly independent of the underlying network support library.
If you want to inject data to RabbitMQ, or want to implement consumer and subscriber for your back-end RabbitMQ then using PIKA would be the perfect choice. It's fairly simple API saves lots effort. Also nice tutorial can be found in RabbitMQ Tutorial section as well.

Paramiko

Again it's better to quote from the official site:
Paramiko is a Python (2.6+, 3.3+) implementation of the SSHv2 protocol [1], providing both client and server functionality. 
So it's all about SSH. If you want establish the SSH tunneling then it's definitely what you are looking for.  Please checkout detail at http://www.paramiko.org/.

Psycopg2

This is an awesome library to play with your Postgres database. For one of my projects, I needed to work with our database which is in Postgres from Python. Then I found it and it's actually very easy to use. Official documentation: http://initd.org/psycopg/

Sample code to create database from python using  Psycopg2 would be as below:

from psycopg2 import connect
from psycopg2.extensions import ISOLATION_LEVEL_AUTOCOMMIT

con = connect(database="postgres", user=username, password=password, host=dbhost, port="5432")
con.set_isolation_level(ISOLATION_LEVEL_AUTOCOMMIT)
cur = con.cursor()
cur.execute("CREATE DATABASE " + dbname)
cur.close()
con.close()


subprocess

This is a python library which allows you to connect to process pool, trigger process, get output from the process, pipe to other process and so many other things to with processes. In one of my projects I needed to trigger java in remote over SSH. I did it with help of paramiko and subprocess. Check out detail of this awesome library in https://docs.python.org/3.6/library/subprocess.html

python-datemath

So, one of the test engineers from our testing team approached me with a requirement for our api testing tool. She asked for a feature to do the date arithmetic, For example, adding 1 day to current date or deducting 3 days from current date. 
Doing it using timdelta or python date time library would have been bit complicated. But python-datemath actually saved me from lots of hassle! Github doc actually described the usage quite nicely to start with.

Apart from those, I have worked with other libraries as well, namely json, re, psutiltraceback,simpleeval and few others. I am still learning and exploring new libraries, because I am focusing on Python nowadays and I am enjoying it. Probably in future will discuss about new packages.

Questions you should probably ask as a QA

During the journey as a QA and Testing practitioner, came across with lots of talented QA engineers. Some of them are excellent as manual tester. I admire their thinking capability, ability to explore scenario and killer attitude towards to discover the bug. Another group I met who are solely interested for automation. They are quick and efficient in design and development. I learned and still learning from most of them. But.....

I found one thing is missing badly is the habit of asking questions. I prefer to ask questions, regardless of the client, product owners or developers. It's true that in most of the cases I don't deal with the clients, and the role I am playing now as architect, the scope of asking question is bit limited. But....

I think QA and testers should grow the habit of asking questions, at least  following are some important questions I can think of:

Is this the appropriate technology for developing this?


I completely agree that this decision should be made by the analyst or architects of the team. But as a QA engineer, I need to be aware of the features, advantages and flaws of the technology that we have picked for the development. This will also help to explore test scenarios to increase test coverage. 

For instance you are testing an app with MSSQL as database. Try out scenario to insert unicode or double byte characters into database from front-end. If the column not properly configured as nvarchar, may be it'll not store the data as expected.

Is there any usability analysis done for this feature/story?


This is very important. Personally I am a big fan of usability analysis. Unfortunately, most of the organization I have worked with, usability analysis and testing are not QA thingy, product owners or BI associated to the feature or story are doing that. But QA should take few moment to understand the workflow from usability perspective. Try to measure how simple the feature is to learn and use. If it's difficult for me to use then definitely it would be hard for end users to use as well. So raise your voice for the usability.

What value this requirement will add?


May be it's not directly impacting your testing, but good to know. Being a QA is like the core of the operation. You should have complete idea of the business. You should be able provide your insights on the feature, who knows, may be it will help the analyst team to revisit the story and reshape it! It's also a good way to increase your habit of asking the important questions.

Who are our competitors and in what extend we are ahead of them?

Hmm, now why I am emphasizing on competitors here? Learning about competitors is like learning about new products as well. For example, I am working with payment gateway now, I am interested to know about our competitors so that I can learn about their business model, strength of API and later can compare and help our analyst to explore gaps within our product. I agree that in reality, mostly it's difficult for me to do research on this and know our competitors. The way we are doing it is by reading contents shared by our Business Analyst team. Please find the best way which suits you 😊. 

Is our current testing procedure able to ensure maximum testing coverage?

This is very important. Answer of this question should be result of ongoing review of the current process. But first of all, QA or QAs of the team need to raise this question. Need to discuss within the team, gather feedback from other teams, analyze test results, bug reports and find answer of this question. Again, it's upto the team culture how you are gonna work with this, but raise it.

Do we have any infrastructure for performance testing?

Importance of this question is actually depends. It depends on the nature of the business. For example, ours is a payment gateway where we have the need to measure performance stats based on user counts, concurrent access and volume of the requests. So we are building our performance testing infrastructure and working close with DevOps and Architects to improve the performance. Look into your project/product, find if this question is applicable for you, if valid then work on it.

Why you followed 'X' way to implement,  where 'Y' seems much better?

This question is more related to the habit of reviewing developer's code. If your testing team has the practice of doing that and you are sure you would be able out smart them by asking this question with proper logical explanation backing your question, then go ahead, shoot it! It's also a good practice to nurturing  our technical thinking and help us to follow developer's mind. It's a good way to argue with developers with developer's language.

Can we automate this feature?

It's a very good question. I really recommend to ask it as much as possible. Prepare yourself and your team in such a way so that any feature which could be a regression candidate in future, could be automated by spending minimum time and effort. 

Why there is no bug?

Fishy right? Bugs are the outcome of your outstanding testing. If there is no bug after your testing, that means either development was awesome or something went seriously wrong during your test execution. So it's time to ask this question, look back into entire test execution process and catch the bug!

These are just few cents from my experience and observations. May be few are valid, few are invalid depending on the context, but let's ask one or few of these question, I am sure that, it'll help to explore alley on to your QA road.

Happy testing 😊

JMeter - Challenges and my resolutions

Last few weeks I have been working with JMeter, to develop one performance testing framework. Purpose of the initiative is to test our cloud infrastructure with certain amount of load. When I started the project, there were roadblocks, challenges and needed to invent new ideas. In this post I'll describe few of the challenges and resolutions I implemented to overcome those obstacles.
  • Data Driven Test - To inject JSON requests to our API requests, initially I started with hard-coded data in Body section of the HTTP request sampler. Then realized, hey, if I feed these data from file, then it would be easier for me to reuse same for different plan as I'll be testing same API for different test purpose. So,
jemter_req_1
became

jemter_req_2

_FileToString is the function I was looking for. I wanted to read content of the file and post it during API call. So here is function to do that. But hang on! it's not replacing ${__UUID()} part with unique id!
  • Function call and variables - That problem of UUID led me to another obstacle. It was literally a roadblock for me. But thanks for having Stakeoverflow in our life :). Initially I though to use beanshell preprocessor to read content of the file then replace the value from code. Then I came across this thread in Stakeoverflow, which solved my problem. I wrapped FileToString call with eval as follows:
    ${__eval(${__FileToString(${__P(dataDir,'')}/data.json,,)})}

So I know how to work with function calls and variables for the content fetched from file :).
  • Reuse Samples: I am not 100% sure whether my solution is proper or not, but at least this solved my problem. When I was designing my tests, then I realized there are few API calls I need to make multiple times. So I started to explore way to reuse JMeter samplers, like HTTP Request. So steps to have reusable controller is as follows:
    1. Add a Simple Controller from Add > Logic Controller > Simple Controller to your thread group.
    2. Then add the HTTP Request sampler under that simple controller. This HTTP Request will be used to send request to the API.
    3. Now add another Simple Controller (as we did in step 1) into your thread group.
    4. Add one Module Controller (Logic Controller > Module Controller) under the newly added Simple Controller.
    5. Select which controller you want the module controller to load for you.
reuse_controller

That's it! Now you should be able to reuse same HTTP Request as much as you want.
So far that's from me for now regarding JMeter, I'll be updating this post with new challenges and solutions because I am sure that a number of obstacles are waiting for me ahead in the alley :).