Spotlight: Performance Engineering Testing from the Cloud

This is a follow-up to our prior Spotlight on Performance Engineering, Performance Engineering and the Chaos Monkey. A key part of performance engineering is performance testing. The Cloud is also influencing where we test from, options for location of the team are changing. The majority of tests are still done from within the Enterprise, usually in a large room or lab. The Performance team may be in one building and the system under test, in another time zone, within the Enterprise. The Cloud changes this by allowing the team to be anywhere, when designing and executing a test. The testing team should be as close to the application development team as possible to help support test execution and help triage.

The entire testing infrastructure is in the Cloud. The test controller or test manager is in the cloud, the test load generators are in the cloud, monitoring tools are in the cloud and the results reporting and results repository are in the cloud.

Collaborative is designing and executing performance tests using the SOASTA Cloud test suite and our Amazon Cloud. Still, the performance team is working in the Enterprise, or from our offices, during the project. Our team goes through the planning, designing, execution, analysis, and results reporting process during the project. They provide critical information to the architecture and development teams. They help identify performance issues within the application and help to recommend changes to improve the application performance.

During one of our recent projects, we had an interesting change of venue for the testing team. The team lost its Internet connection, the ISP went down. The ISP was not able to provide an uptime for when the access to the Internet would return. When you are working on a Cloud based performance engineering project, you suddenly realize how important that is.

The Sr. Performance engineer on the team was able to come up with a solution to get Internet connectivity. There was a nearby Au Bon Pain with an Internet hotspot. They moved the testing location to the café. From here, they were able to access then Amazon Cloud services and continue running the performance tests. We are now adding coffee and croissants to our performance testing toolkit.

This is an overview of the methodology our performance engineering team uses for our performance testing projects...

Collaborative’s performance engineering team can bring their expertise to help your team meet its goals for performance, scalability and stability. This allows your team and your scarce resources to stay focused on designing and build the application. We work hand in hand with your development teams to transfer application knowledge, allowing us to operate more independently. We also provide best practices and guidelines for designing and developing web sites to provide your customers with an outstanding experience on your web site.