Blogs from Kiran Badi

Few Days back I was a part of the debate with few ASP.NET developers where we were discussing the pros and cons of various web servers available in the market and types of authentication methods supported by them, why quite often public facing sites uses form based authentication and not any other types of authentication.I have been the fan of IIS since its 5x days and have learned a quite a bit about it from IIS product team members.( I am also huge fan of Apache Webserver, only think I hate about them is that they stopped giving binaries to Windows Users and its pain to built it from source unless we are on .nix platform).

I call Authentication as something where users supply their credential in order to identify themselves to the server and servers based on the roles configured in some system it determines their authorization powers as what type of operations users are entitled to do in the application.Authorization always happens after Authentication and almost at the same time.

Latest version of IIS provides around 6 ways for doing authentication and they are ,

• Anonymous Authentication: In this type end user do not supply credentials, effectively making an anonymous request. IIS 7 impersonates a fixed user account when attempting to process the request.This type of authentication is mostly used for public-facing web sites where visitors are not required to supply credentials.Users can access the site freely and browsers will not prompt the users for any kind of challenge. Its enabled by default in IIS 7.0 version. One of the best way to prevent users from accessing any resource which requires credentials is to create separate group and assign appropriate  permission levels to it. Also execute permission levels should be denied to Anonymous users on windows directories.
• Basic Authentication: In this type end users is prompted to supply credentials, which are then transmitted unencrypted in base 64 format across the network. Basic Authentication is supported by all major browsers. This type of Authentication should be used only when traffic flows entirely on SSL so that the data flowing in the wire is encrypted.Anonymous Authentication should be disabled in case we want to use Basic Authentication.
• Digest Authentication: Over here end user are prompted to supply credentials, however not like in Basic authentication, the user’s password is not passed in clear text across the wire but it’s hashed using MD5.It’s mostly used along with Windows Domain Controller.Browsers needs to http 1.1 compliant in order to use this type of authentication.In addition to this Anonymous Authentication should be disabled. Quite a few intranet application in the large companies uses this type of Authentication internally for their application.
• Integrated Windows Authentication: It contains two separate authentication schemes: NTLM v2 (NT Challenge/Response) and Kerberos. Enabling Integrated Windows authentication using IIS Manager enables support for both of these two schemes. NTLM works similar to to Digest authentication (it hashes users password). Kerberos relies on shared secrets between the client, ADC, and the IIS server to authenticate the user. Kerberos is only available for Windows Active Directory accounts, whereas NTLM can be used for local accounts as well. IIS 7.0 does not present Kerberos as a discrete authentication option to the client, instead sending a “Negotiate” option, allowing the client to choose Kerberos or NTLM. NTLM can be presented as a discrete authentication option to the client. Microsoft recommends using this type of authentication for Intranet Applications since client and servers share the same domain. This type of authentication is not useful for internet because there is no encryption in the internet.
• Client Certificate Authentication:When using this type of authentication, the client presents a certificate to the server. The server is configured to map certificates to one or more Windows user accounts (it is possible to map multiple certificates to a single user account or to map each certificate to an individual user account). IIS logs on the mapped user account.Client Certificate authentication requires that SSL/TLS be enabled for the resource being secured.Mapping client certificates lets you automatically authenticate users who log on with client certificates, without requiring the use of other supported authentication methods such as Basic, Digest, or Integrated Windows authentication.
• UNC Authentication : When Server needs to retrieve files from a remote network resource e g file share, a virtual directory in IIS can be mapped to a UNC path. When configuring this virtual directory, it is possible to specify a some fixed user account that will be used to connect to that file share, irrespective of the identity of the end user.
• Form Based Authentication:It relies on the supply of credentials via html form as part of the HTTP traffic.In this way, the request for the login form is an anonymous request. After authenticating via the HTML form, an authentication cookie is set by the server.The client must return this cookie with each subsequent request in order for the request to be authenticated. Although this authentication can be configured using IIS Manager, it is effectively ASP.NET’s Form Based Authentication.Forms Based Authentication can be combined with either ASP.NET’s authorization features
  (available with previous versions of ASP.NET) or IIS 7.0 new inbuilt URL Authorization feature to protect access to resources.

All these types of Authentication can be configured at directory level, website level or at file level. Maybe sometimes later I will write some posts as how to configure these types of Authentication in IIS.

One of the hard things of troubleshooting the production issues is to identify the root cause of the issue. When the application is deployed in the open internet and accessed by the variety of the browsers, and has large number of hops in the network, it becomes quite a challenging task. So in this post I will walk you through the list of things which needs to be checked at high level on client side in order to identify the root cause of the issue and before proceeding to do the review of the code base.

Browser Setting

• What is the proxy setting of the browser, does it access the application via proxy or connection is directly to the internet.
• How is browser configured? Does the user have administrative rights or he has basic rights on the browser settings.
• Is the browser configured to show user friendly messages? If Yes than can we reproduce the issue by removing those user friendly messages and just displaying the actual message which application is throwing.(Please note if your application does not do proper error handling, there exists a risk that you are displaying nasty code to the users. E.g. Famous yellow screens of the .net)
• Is the browser configured to run in standard mode or compatibility mode? This point applies to IE. (Ajax and UI Issues are most often related to compatibility mode)
• In case the issue is related to certificate, then checking whether the relevant certificate is present in trusted store often helps.
• If your application uses popup windows to display some information, then checking if there are any add on or setting in browsers which is blocking pop ups also helps. Some browser add on silently block pop ups without giving any information to the users.
• Is the browser setting on default? These are the setting which is factory default. One of the easiest ways to troubleshoot issues is reset the browsers to default setting.

Client Computer Settings

• Is the client computer behind the firewall? If yes than verifying that it’s correctly configured saves lot of time.
• Checking the antivirus software installed on the client machine also helps. Sometimes in case where your application uses specials character, there exists a chance that badly configured antivirus might filter out or block the incoming responses.
• Hardware and software configuration of the user’s machine. In case if your application does lot of heavy lifting on the client side, then it often helps to educate the users that minimum configuration needs to be met.

Network Infrastructure and configuration

• Is the network correctly configured? Using the bidirectional ping command often helps to identify the network issues.
• Is the client able to resolve the application host name correctly?
• Checking how many hops the user needs to make to connect to the server also helps
• In increasing user experiences. General Thumb rule I often use is more the hops user does to connect to the server, more the response time he is going to get.
• Is there any load balancer or firewall between the server and client? If yes than checking if they are correctly configured also helps.

User access or Login Issues

• Is the user giving the right credentials and if yes then checking if server is doing correct validation of credentials also helps.
• Is the identity and access validation done by application or by third party component like Site minder? If by third party, then checking the third party component in isolation often helps.
• Does the user have appropriate access level to access the resources? If yes then further troubleshooting is required. Else its waste of time.
• If browser is client, then disabling friendly error message settings in browser will reduce the time to identify the issue by almost 50% since no extra debugging tools are required unless the case if of missing or stolen headers.

Over the years working in projects and after analyzing logs year after year I have seen one trend which is been frequently repeated by developer or development teams, they just don’t log things about application what they need to log. Since they don’t log information correctly, all activities further downstream become painful and subsequently lead to not so memorable fix ever for a smaller issue.

Now why I am giving so much importance to logging? Probably I love logs and probably I know something about importance of logging and therefore maybe I can share with you as why logging is important, what things we need to log, what logging levels are, how they help project teams and when they become bottleneck.

Log4j is one of most frequently used logging libraries out there and it’s also one of the most robust libraries used for logging. Log4j can be configured via properties file and this is good approach for setting up logging. With log4j, we can log information with quite a few levels of granularity, namely debug level, info level, warn level, error level, and fatal level, all and trace level. Below is my high level understanding of these levels,

• Debug levels are used when we need to debug the issues which are minor in nature and are occurring frequently.
• Info level is used when we need to log events which are significant in nature and are important events in the application life cycle such as initialization of JNDI Resource, Data source etc. Logging all the significant events helps get to information without running into the code. We can also track the progress of the application at granular level.
• Warn Level is used to log minor problems which are more of warnings and these are ones which at times are external in nature. They are used to log potentially harmful events. Example logging the input parameters which are not acceptable to the program.
• Error Level is used to log exception details or errors which break the functionality of the application. Normally errors do not crash the application, however it might break the subset of functionality which generates the error.
• Fatal levels are used to log the fatal errors .Fatal errors are generally related to crash of the application or some components of the application. With fatal errors, applications normally abort working.
• Trace levels are used to trace fine level informational events.
• All level is used to log all the information events. It turns on full logging.

One can choose any of the above levels to log information depending on the priority and severity of the issue. One can also customize and extend the logger class to create the custom level if required. But I feel that would be an extra work and definitely not required since existing levels will provide the information as what we require to confirm the issue.

Logging helps in many ways. Unless we log we will never know as what exactly our code is doing. We will never know the path our code takes to implement the functionality. Logging the information also helps to understand and troubleshoot issues after application goes live. In fact job of the many technical product support people who are not so good in technical aspect of software rely just on logs and provide solutions to the issues encountered by the users. Logs also help in User data mining. Google/Facebook/Amazon extensively logs the information about its users and then uses that information to understand the user’s behavior and then come up with some functionality that delights its users. In short logging also helps to grow the business.

However there is certain information which should not get into the logs, like,

• User sensitive information like password, user ids etc.
• Any personal information that can be used to identify and trace the user and has potential do harm to user in case it falls in wrong hands.
• Any information which is financial in nature like bank accounts, credit cards details etc.
• Logs should not also contain information about infrastructure on which application is running. They should only contain information which techy in nature.

Now let’s talk about some good ways and things that should get into the logs,

• Logging should always include the source of the event, namely class, file or package names that generate the event.
• Logging should always be categorized as info, error or debug as per your application requirements.
• Logs should be rotated always once their reach defined file size or preferably on daily basis.
• Logs should contain timestamps preferably server time stamps.
• Logs should be human readable and parse able so that information can be extracted from it easily.
• Logs should be stored preferably on local machines.
• Logging should contain application start/stop details, significant events related to application life cycle etc.
• We should also log details of events like how much time it took to connect to database, how much time it took to execute query, how big was the result set etc.

Each of the information in the logs should always contains details as who (Class), when (timestamp), where (Which part of code), what (what action did code do) and finally output that gets generated for the action. Some of the folks I know who built codebase that processed millions of requests in an hour had used logging extensively to fine tune to their code base. Once their code base was tuned appropriately, they turned off the logging and then deployed it in test/or live environment.

There are also few things which are not so good about logging which we need to understand,

• Excessive logging leads to bad performance of the application especially when logs are written in network drives. It chokes the network.
• Using incorrect logging level for getting information is also bad practice.
• I personally prefer to log information asynchronously as this will not block my code execution. (Take this suggestion with pinch of salt as I am still implementing logging for my code, so this statement might be wrong, but I know we can log in async mode, however it also depends on our code, it should allow multiple threads to run).
• Logging on the same local drive as server often has IO overheads. Think as what happens when we have 100’s or 1000’s of users are on application and they all are logging.

Though lot many experts believe logging do not take more than couple of nanoseconds to couple of milliseconds to log information per user, however based on my experience, people implement logging very poorly, they just don’t log the information what is required or do excessive logging for various reasons or log it sync mode. Under load test based on my experiences, application which has got info level logging will have at least 4 to 10% more response time than when it has got logging level as on error. Of course this statement is true  if developer has some logging set up else expect that its going to long journey to with steps something like setup the logging infrastructure,reproduce the issue, verify the issue in logs, check the code for execution path,confirm the issue in code and then provide the fix for the issue.

Recently I inherited some of the  LR scripts from one of my colleagues,it was all about building the json calls for stressing the backend spring security framework which was first layer of entry into the mobile infrastructure.Those scripts were simple scripts  built using the custom request with json string as a body part.One of the things that really surprised me as part of this effort was that web custom request in itself was taking close to 100ms to 300ms to do decompression of the server response during load testing.

Okay first let me give you some background,servers were configured to send the response compressed in gzip format with content encoding header as gzip.The functionality under scope had SLA of 1 sec max and quite a few functionality in scope also had SLA that was less than 500ms.Quite a challenging SLA’s I would say.But again these functionality were supposed to be accessed over the mobile device,so probably less the response time better it is for users.

Most of the response coming from the server for all functionality was served as chunked bytes,so what it means is that server sends initially some bytes as response in compressed gzip format,LR decompresses  those bytes in 5 to 10ms and then again server sends next range of bytes as chunked gzip response and then again LR will spend close to 5 to 10ms to decompress those bytes and like wise the process continues till we have final set of bytes.All these process happens in the single connection and connection never closes with the server.In case if you do have some server response validation in place, then expect that it will add another 10ms to do that validation.

Now I have measured all these times in the single iteration of vugen,these times increase exponentially when we are running the Load Test in controller or PC and this overhead of decoding the gzip content becomes a quite an issue when response time SLA are in ms.

Here is how it looks when you see the behavior in LR Vugen with decompression on in the script.You can see that it takes 5ms to decode the 154 bytes of response.Now imagine the normal webpage will have size of 2mb of data gzipped,so you can see the impact of this decoding  when size of page increase specially when response is coming as chunked bytes with no fixed content length from the server.

I think HP LR team might also be aware of this behavior and probably that the reason as why they might have come up with function to disable this.Use Web set option with decode content flag turned off if you are running the scripts which do not require validation and has response time SLA’s in ms.The drawback of disabling this feature is that all your correlation and other checks for server response will fail since server response will show up as binary content like below.

I would suggest you to disable this feature if you can and do the response validation by using the other techniques like verifying server logs etc.By disabling this you will gain close to 15 to 20% reduction in response time reported by LR.

Is this expected behavior of LoadRunner ?, I think they have to do this,unless they decode the response, none of the other function like web reg save param or web reg find will work and these functions are core functions of LoadRunner.Probably the right way is that LR should not add these decompression timing in their transaction markers.These timing really pollute the results specially for web applications or probably they can increase the speed of this decompression library what they are using in LoadRunner.

Recently I have seen lot many people complaining about the performance of the Indian Railways website and most complains relates to the fact they are not able to book the tickets online and it takes hours for them to book the ticket via IRCTC.I also had faced the similar situation many times but I must say I have been fortunate enough that more than 95% percent of the time,I was able to book it in the first attempt.Maybe its just my luck or my good timings.So I thought let me do some investigation in my spare time and see how much traffic does this site handle and what all are the real trouble the users are facing while using this site.So with this intention,I pulled out the below facts about IRCTC site,(Please do remember that Government employee’s rarely exaggerates the numbers as they just don’t have any motives to play with numbers,based on this wisdom, I believe these below numbers are correct or should be on lower side),

• IRCTC site receives close to 10 to 12 lakhs(10,00,000 to 12,00,000) hits per minute.
• The site uses around 450mbps bandwidth.
• Nearly 5 lakhs tickets are booked on daily basis.
• At any point of time, it has got close to 10,00,000 concurrent connections open to it.

Of course IRCTC has taken some measures to increase the scalability of its site by adding flash memory drives,restricting the activities of its agent and ensuring that it does regular maintenance of its infrastructure,but I do feel personally it still needs lot of improvement given the kind of traffic it gets.

I can understand general users who don’t understand the complexity of the software complaining about it,but I just don’t understand as what makes IT Folks who had spend years designing IT Systems/Testing to complain about this site,can’t they understand the complexity/volume of the traffic the site handles and the way bureaucracy is slowing it down or they believe these numbers are negligible in volumes.

I feel the folks who are working on this site and folks who had scaled this site has done remarkable job to the people of India.They are in fact heroes and they have wealth of technical information with them. So I would request people who run technical IT magazines contact these folks and maybe you can ask them as what is secret of their scalability in spite of working in tight/constrained and limited work environment.How on earth can they maintain close to 10 million concurrent connections ? Isn’t this a interesting case study worth a attention ?.

Source for this article are here, here,here and here .

Recently I came across this study from ladders which tells us that recruiters take just 6 seconds to scan the profile.The study was bit interesting and gives interesting findings.6 seconds to scan the profile. I believe 6 seconds  is close to nothing and it  means that you flip over the pages and you make a decision whether to forward the profile to hiring manager or not.

I don’t know why but I feel there is some kind of relation between this 6 seconds profile scanning and quality of talent we have today in this market for IT Profession.The first thing that comes to my mind when I went through this article was if recruiter takes 6 seconds to make the decision , then obviously the hiring manager might be taking the close to 20 min max to move the candidate to the next round, or may be company might be using services of the person who might not be qualified to take interviews.In all it looks more of the scam then real hiring process where in recruitment team/Practice team/Account team align together for their motives.

I am aware that ratio between vacancy to be filled and persons applying to the that position is often in the ratio of 1:50 or at times more than that.However this do not mean that we take just 6 seconds to scan the profile and make a decision.Job of the recruiter is ensure that right candidates with good profiles are given a chance to show case their skills and with 6 seconds all you can do it scanning the profile with some keywords and if those keywords match, give them a chance.I feel this is terribly flawed process of doing things.

I am also aware that most recruiters are not familiar with the technologies used in the field and they believe that with scanning keywords they can select the right profiles,No its not the case, there exists many other ways where in you can choose the right profile some of which I can think is as below,

1. As soon as the recruiter gets the mail from account or practice team that we need to hire people, they should talk to the concerned team who has raised the request and get more information from them as what kind of people that are looking for, what kind of environment the project is in,how big is team going to be, etc. etc..Keep in mind that working exposure on projects is different that working exposure on Programs or Portfolio of projects.Though technical skills required are the same,but pressure to deliver is more on programs and more flexibility is required in programs.Setting up 30 minutes meeting would be right step.

2.Recruiters should also discuss with team as what kind of technologies they are looking for.Ask them to give the description in simple language.If you still don’t understand or if they throw technical jargons in JD, then probably you need to tell them to make it still simpler until you understand it correctly.

3.Work closely with the hiring manager, before making the first call to the candidate, ensure that you have discussed his profile with hiring manager so that we know that we have chosen the right profile to take it forward.

4.It’s the job of the recruiter to go through each profile irrespective of the number of profiles received, if you alone cannot do it, then probably you need to seek help.There is nothing wrong in asking for help.If someone denies you or assumes something about about you, then probably you are asking a wrong person and at that time you need to escalate it.Keep in mind, its all about team work.

5.I have seen lot many recruiter who cannot push back to the business managers/practice teams,I feel you need to learn this skill of pushing back.Its required at times to push back for various reasons.I don’t know but I have seen most recruiters feel their only job is keeping hiring team happy by sending them lot of profiles for each of the position.I don’t think that makes anyone happy, we look for good candidates with right profiles.If you send lot many profiles and if those are rejected , then hiring team will surely have negative impression on you.By using the process of keywords,you are increasing your chances of making this mistake.

6.You should also understand that professionally written profiles contain more make up than reality for various reason though I know not all contain makeup.You can always ask the candidate to correct the profile and ask them to resend it.With globalization, there are lot many people in market who might not be familiar with written English completely.However please note that some jobs require proper written English, for those jobs, feel free to reject profiles.

These are some of the things that comes to my mind and do understand that we do appreciate recruiters who help us in getting right candidates.I am sure candidates are also thankful to those recruiters who help them in getting jobs.I still remember all the recruiters who had taken my profile and made an attempt on my behalf.I am thankful to them.

In this post, I wanted to share some tips and also clear some misunderstanding which I have seen in lot many performance Engineer’s with regard to JavaScript URL’s which we often see during the Load testing of the Rich Client or web based Application which heavily uses JavaScript for rendering and manipulating the User interface.

I am sure lot many of us has seen url’s like below during their load testing efforts,

/mysite/js/myjsfile.js?1357011161255

The above url contains the unix style timestamp appended towards the end.There are quite a few reasons as why we append the timestamp the JavaScript,foremost reason being that we do not want to cache the JavaScript.Since these scripts are often interacting with DOM and building DOM Elements on fly based on the user interaction, it is a good thing in certain situations that we do not cache these type of javascripts.If browser caches these scripts, there exists some risks that we might see some browsers quirks happening at the UI.

Secondly, most of the JavaScript libraries which provides and implement XMLHTTP Requests implicitly uses these timestamp features.I know DOJO Library and Jquery extensively uses these timestamps while making Ajax calls so as to prevent the caching on the user’s browser.

One can easily implement these timestamps in their load testing scripts.I know both SilkPerformer and LoadRunner has built in Functions which supports creating and replacing the unix style timestamps.I suggest where ever you see these types of URL, just use those functions ,rather than commenting out those URL’s.If you are commenting out those URL’s, I feel you are building the scripts incorrectly and downloading at least 15% less bytes when compare to size of the entire page.

Also when you comment out these types of URL’s, you are doing less calls and thereby reducing the load on the servers.We need keep in mind that users never comment out anything and for these types of URL’s whenever they are present,browser will always be forced to make the call to fetch these JavaScript URL’s.There might be some performance impact on the application but again its choice between providing functionality and achieving performance.

However if you are developing Rich Client Web based application, then I suggest you to append your JavaScript files with timestamp in at least your development environment.It saves lot of time in debugging various client side issues and relieves you from clearing your cache every time you compile and build your code base.We need to keep in mind that browsers almost always caches JavaScript and css files and updated files are not available to application unless we clear all browser history and close and reopen all browser windows.Quite a painful process specially when we are implementing multi page functionality.

Today I decided somehow to take some time out for writing something as what I did in 2012 and make a short note as what I missed and what I achieved.

2012 was an very exciting year and maybe the year which will go down in my life as an year where I gave up lot of things for various reasons/circumstances and an year which gave me strength and complete different perspective of  my abilities and deficiencies.

Some of the things which I did and I missed in 2012.

• For quite some time now I was owning bunch of domains.Out of those bunch of domains,I picked one and created my own personal site.It’s a 3 page site.Its personal site and gives some  information about me.Its good that we have some internet presence.I feel everyone should have one.
• I started believing more in chemistry than maths.I said no to someone with whom I really wanted to get associated with.Sometimes very long past forgotten tasks come up suddenly and screw all the plans.
• I wrote lot of code.Now I know how people build applications,deal with scalability issues,how to view the problems and how to solve them in shortest time.I wrote code for image galleries,sending emails,group mails,integrating maps,captchas,building dynamic lists,generics,collections,than for all regular functionality like insert,update,delete and create with regard to UI and databases.This is only thing I am really happy about.I am sure this will come to my help at some point in my life.
• Lot of good folks who participates regularly in Stack exchange/Code Ranch and JQuery helped me in clearing some of my chronic understanding issue with regard to the way algorithms and codes are built and written.Without them I would banged my head at least 10 times to the wall.Actually before going to them I banged my head to the wall at least 3 times with sheer frustration.I am thankful to them.We should always thank the person who points us to right direction even though at times the person flames us.
• Probably this was year when I cried after a very long time with real pain and feelings when my Dell Vostro’s laptop hard drive crashed.I had all my code in it and I also did not have a  backup for it.This incident was one of most  valuable lession learned by me and after this incident I realized why file sharing services  companies like Google drive or Dropbox are hugely success full. Always keep a back up of the work you do and also never show this kind of emotions in front of wife,it creates misunderstanding.
• I will never buy gadgets from Dell going forward.They make sexy and attractive laptops with cheap parts in it.Sorry to say this but that’s my experience.
• There was nothing much exciting about my work.All regular work which I have been doing it for years.Though I found lot of performance issues, there was just one thing which I believe I need to focus on.CSS Property overriding effect on performance and user experience when we have 1000’s of lines of html/js and css code.This is interesting area of performance improvement which I believe is hugely neglected by our profession.
• I did some research on as how IT Industry generally work,how people over here make decisions,what motivates them,how they rate people,and why they rate people like that,what can we do to improve this profession overall.Why some folks in this industry are forced to move out or forced out and where they often make mistakes.May be I will write some short notes on my observation without bias.
• I missed my brother in law’s wedding something which I really really feel bad.
• Then there lot of technical stuffs I attempted and experimented with regard to various front end technologies like CSS/JS and backend technologies like Tomcat,MYSQL. I know I have not written anything in recent months.I will start writing again.
• Then as every year, I watched lot of Chinese,Hindi,French,British and American Movies.I just love watching movies first day first show.

Finally I would like to thank all my friends  and all those companies who offered me an opportunity to write/do something.I really appreciate your offer and hope that some day we might partner.Thanks everyone for your time to visit this blog and Wish you a very happy new year 2013 and merry Christmas.

From past couple of days I have also been playing around with MYSQL 5.5X Database doing some bit of writing queries, creating tables, indexes ,routines here and there for one of project. MYSQL database seems to be bit easy to understand and provides almost all similar features as provided by MSSQL or Oracle. (Of course there might be some difference in the ways they are designed or in the way they use SQL).

As soon as someone reports that application is slow or during test if we find slowness, the find thing we need to do is to identify cause of slowness (Most people don’t do this step, they become defensive, at times even I have exhibited this behavior,its humanly approach). There could be many ways to identify the cause of slowness and there could be many reasons for this slowness. However for the sake of this post let’s assume that we have identified the slowness as MySQL database and we have ruled out other causes for this slowness.

In order to identify the slow running MySQL query, one can run the below command in MySQL workbench or via MySQL client and see whats going on in the MySQL box,

Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current input statement.

mysql> show full processlist\G
*************************** 1. row ***************************
     Id: 1
   User: root
   Host: localhost:51606
    db: mydb
  Command: Sleep
   Time: 372
  State:
   Info: NULL
*************************** 2. row ***************************
     Id: 2
   User: root
   Host: localhost:51607
     db: mydb
Command: Query
   Time: 58
  State:Query
   Info: SELECT * FROM MYTABLE WHERE auto_id = 46102

 

As you can see from above that Select statement in itself is taking around 58 secs to execute.In addition to above,Show Process List command can also be used to get  insights as which threads are running in MySQL server and it is quite often used to debug connection issues.This link will provide more info about this command.

Once we know which SQL is taking more time, then the next task here is to replicate the issue outside the application using the same data and same statement but with using  MySQL client. Only when we are able replicate this issue outside application, we can say that issue is with SQL Query and not with any other elements of the environment or application.In almost all cases replication of issue happens successfully.(However do watch out for those smart and excellent communicator DBA, who often share the screen with businesses to show them that in spite of querying more rows of data, issue cannot be reproduced and query executes in fraction of eye blink,in such cases ensure that we use same set of data which is used in application during the time you saw slowness along with before and after row count for the table and also all condition remains the same.)

Moving on, once you are able to replicate the issue, the next step is to identify the Query plan generated by the query,in MySQL server, this can done  by using Explain Statement,

MySQL> EXPLAIN SELECT * FROM MYTABLE WHERE auto_id = 46102
           id: 1
  select_type: SIMPLE
        table: MYTABLE
         type: ALL
possible_keys: NULL
          key: NULL
      key_len: NULL
          ref: NULL
         rows: 47890
        Extra: Using where

In above query execution plan,any query that do not use an index signified by the key row above in the preceding output can be considered a poorly tuned SQL query. The number of rows read in evaluating this SQL statement,is as signified by the rows row,gives some indication to as how much data is read and can directly correlate to the amount of time required to execute the query. The type row with a value of ALL is also an indicator of a problem.

Adding the indexes to the table might help in these cases,but again it also depends a lot on the structure of the table, so before applying any fix to the situation, it makes more sense to understand the table structure and amount of the data the table holds,

Below command will give you the information about table structure,

SHOW CREATE TABLE ‘MYTABLE’ ;

The above statement will provide you the information about the table along with all column information.Once we understand the structure of the table it becomes quite easy to apply and try out various fixes.Below command will give you information about data length and other various table information

SHOW TABLE STATUS LIKE 'MYTABLE'

Both the above commands gives us very interesting information and this information can help in doing sizing of the databases along with capacity planning.

Once we have all these information, we can start working on applying fixes.Maybe after I fix some of my tables, I can write some more interesting things to do.

Quite often while coding application which has lot of forms in it, there comes a requirement where in developers needs to deal with back button functionality of the browser.Disabling the back button with Javascript is one of the options used by many sites to deal with duplicate submission of forms.

Browsers maintain information about pages visited in the browser’s history and Javascript can be used to manipulate the history using windows.history object.

Some of the methods which we can use to know more about history are.

window.history.back();

This works exactly as back button of the browser.Goes 1 page back.

window.history.forward();

This works as exactly as forward button of the browser.Goes 1 page forward.

The number of pages in the history stack of the browser can be determined reading its length property,

window.history.length

window.history.go(-1);
window.history.go(1);

• window.history.length: Returns the number of entries in the joint session history.
• window.history.state: Returns the current state object.
• window.history.go(n): Goes backwards or forwards by the specified number of steps in the joint session history. If the value you specify is zero, it will reload the current page. If it would cause the target position to be outside the available range of the session history, then nothing happens.
• window.history.back(): Goes backwards by one step in the joint session history. If there is no previous page to go to, it does nothing.
• window.history.forward(): Goes forwards by one step in the joint session history. If there is no next page to go to, it does nothing.
• window.history.pushState(data, title [, url]): Pushes the data specified in the arguments onto the session history, with the given title and URL (the URL is optional).
• window.history.replaceState(data, title [, url]): Updates the current entry in the session history, with the given data, title and URL (the URL is optional)

History object of HTML5 gives us the tool to push/replace the url in the browser’s history and its this feature which I believe is somewhat in secure in nature.Maybe we can have some warning message whenever some scripts wants to read or write the history of the user’s browser exactly the way geolocation api’s work.

Anyways coming back to our topic, we can also use the javascript to control the behavior of the back button,one of the ways quite often used is

function disablebackbutton()
    {window.history.forward()}
disablebackbutton();
window.onload=disablebackbutton;
window.onpageshow=function(evt){if(evt.persisted)disablebackbutton()}
window.onunload=function(){void(0)}

This is most ugly way of dealing back button problem.Just disable it on onload and onpageshow events.

Another clean way of dealing with back button issue is tell browsers not the cache or store any information of the page in its history and this can achieved by setting appropriate headers.With Servlets and JSP , this can done  by

    response.setHeader("Pragma", "no-cache");
    response.setHeader("Cache-Control", "no-store");
    response.setHeader("Expires", "0");
    response.setDateHeader("Expires", -1);

These headers can be added to any page which requires that it should not be cached or reused in any way.I did go via header’s approach did resolve the back button issue.Now whenever I use back button I get Page expired message and for some pages form values are not pulled out from cache.

I would suggest going via header’s approach as this can be done at server side and has very limited or low dependency of the clients and most browsers honors those headers.