In my tenure of working as a performance engineer for over half a decade, I have worked with architects, senior devs, and leaders in setting up infrastructure for conducting enterprise-level load tests, and often I have realized that every infrastructure and every setup is unique in its’ own way.
Recently, I have been collaborating with a client whose scripts possess a high level of complexity. To facilitate efficient performance testing, we are implementing Neoload 2024.3, which offers innovative features to delve deeply into server-side metrics, particularly when 300 to 500 VUs are accessing the system.
This post is a reflection of first principles I have already written about in my previous blog posts regarding performance testing tools. The link to the posts is at the end of this post.
The following content will be covered in the following post:
- Understanding NeoLoad Architecture
- Hardware and Sizing Considerations
- Installation and Configuration
- Security and Network Considerations
- Monitoring and Observability
- Cloud Vs. On-Prem Load Generators
Understanding NeoLoad Architecture
1. Controller: Manages the test design, scenario configurations, and orchestrates the load generators. It is also responsible for aggregating and analyzing results in real-time.
2. Load Generators (LGs): Execute the virtual users (VUs) and generate load against the target system.
You typically have a single Controller (or multiple Controllers in a high-scale, distributed setup) and multiple Load Generators to simulate large numbers of concurrent virtual users.
Hardware and Sizing Considerations
Controller
CPU: Aim for at least a 4-core or 8-core CPU (preferably 2+ GHz per core). NeoLoad’s real-time analysis and overhead require enough CPU resources to handle test orchestration and result aggregation.
Memory: 16 GB is usually sufficient for most moderate-scale tests; for large-scale tests (tens of thousands of users or complex test scripts), 32 GB+ is recommended.
Disk: Use an SSD for faster I/O. The controller logs and analyzes a significant volume of data during and after tests.
Network: A stable, high-bandwidth network connection is crucial for data exchange between the controller and remote load generators.
Load Generators
CPU: Each LG typically needs at least a 4-core CPU for moderate tests. The more complex the scripts (e.g., handling multiple protocols, JavaScript, etc.), the higher the CPU requirement.
Memory: 8–16 GB is common for load generators, but the actual requirement depends on the protocol and complexity of the scripts. If you’re testing on real-browser, memory consumption can increase.
Disk: Typically, LGs don’t require large disk capacity—SSD is helpful but not mandatory. Ensure enough disk space for logs (especially if you enable advanced logging).
Network: LGs must have sufficient upstream bandwidth if you’re sending large requests or testing streaming protocols. Also, ensure low latency between LGs and the target environment if possible.
Installation and Configuration
Operating System
NeoLoad Controllers and Load Generators can run on Windows or Linux. Choose the OS you are most comfortable managing and monitoring.
On Windows, ensure that User Account Control (UAC) is correctly handled and that the user running NeoLoad has administrative privileges if needed.
On Linux, ensure correct file permissions and that any dependencies (Java-related or otherwise) are met.
NeoLoad Versions and Licensing
Make sure your Controller and Load Generators are on the same major version of NeoLoad (e.g., 7.x). Mismatched versions can cause compatibility and stability issues.
Have the correct license file (or license server configuration) in place on the controller to accommodate the number of virtual users you plan to run.
Controller Configuration
Configure result storage options (e.g., local storage vs. external DB) based on test data retention and analysis needs.
Adjust the analysis settings (in the Controller’s Preferences) if you notice performance constraints during large tests; for instance, limiting the amount of data retained in memory for live analysis can help.
Load Generator Configuration
Java Settings: NeoLoad uses Java under the hood. Tuning the JVM options (e.g., -Xmx values) can help if your scripts are memory-intensive.
Network and Firewalls: Ensure the necessary ports (default is 7100–7101 for Controller to LG communication) are open. If running in cloud environments (AWS, Azure), confirm that security groups, network rules, and routing are set properly.
Headless Mode: On Linux-based LGs, confirm if you need or don’t need a display environment. Usually, load testing is headless, but if you are simulating UI-based scripts, ensure the environment can support it (for instance, an Xvfb environment).
Security and Network Considerations
Firewall: If LGs are in a different network segment (e.g., behind a DMZ), open the required ports or set up secure tunneling (like VPN) so the controller can communicate with LGs.
Encryption and Certificates: For HTTPS traffic, ensure you handle SSL certificates correctly. If you are using client-side certificates or custom trust stores, install these on the LG machines and configure them within NeoLoad.
Secure Credentials: If your tests involve login credentials or API tokens, use NeoLoad’s variables/vault approach to store sensitive data rather than hard-coding it.
Monitoring and Observability
System Resource Monitoring: Use built-in NeoLoad monitors (and external monitoring tools) to track CPU, memory, and disk usage on both the Controller and LGs during the test.
Application Monitoring: Integrate with APM tools (e.g., Dynatrace, AppDynamics, New Relic) to understand server-side performance metrics.
Logs: Store logs centrally or in a location that is easily accessible for troubleshooting. NeoLoad logs can be quite detailed when issues arise, so always monitor disk space.
Cloud Vs. On-Prem Load Generators
Cloud LGs: NeoLoad supports cloud-based Load Generators (e.g., AWS, Azure). This approach is quick to scale and is generally used to test external-facing applications. Make sure you properly handle security (VPN or reverse proxy) if you’re testing internal apps.
On-Prem LGs: More control over the environment and network routing but can be harder to scale. Ensure you have the hardware capacity to support your load requirements.
Summary
When setting up NeoLoad Controllers and Load Generators:
✅ Match hardware resources to test complexity and scale.
✅ Secure communication between the controller and LGs with the correct ports and certificates.
✅ Monitor resource usage on all critical components (controller, LGs, and application under test) for bottlenecks.
✅ Plan for high availability: If you run mission-critical load tests, consider failover strategies (e.g., backing up test designs, distributing load across multiple LGs, or using a licensed secondary controller).
References
✅ NeoLoad Official Documentation
✅ Recommending that you enroll in Tricentis Academy to gain a deeper understanding of NeoLoad
Leave a Reply