The test execution information collected in the test planning phase must be transmitted to the test bed so that the test bed operator has all the information needed to execute and track the tests. An automated interface that allows transmission without media breaks avoids error-prone manual steps and reduces the time required for test preparation work.

It is important here that the test order must be interpreted by the test bed automation system in the same way as in the higher-level test process management system. Formats and contents must therefore be coordinated with each other.

A common way for data exchange is to provide the data in files. The test bed and test planning must use the same transfer directory and a common file format for this. In addition, it must be ensured that no "half-finished" files are picked up during the writing process.

Another, often chosen way is also the data retrieval via the programming interfaces of the test planning systems. These are often equipped with a REST interface or allow direct database access to their dataset.

In order to be able to compare test data more easily later and to simplify processing in analyses, data harmonization is often carried out in the course of the data transition from the test bed to the test management system. This involves standardizing different file formats, but also making adjustments with regard to naming conventions, units, etc.

Since large amounts of data usually come from the test bed, the data is usually transferred as a file transfer. During data transfer from the test bed, no data may be irretrievably lost due to system failures. It is therefore important to select a secure transfer mode with confirmation of complete and correct receipt and to temporarily store the data until successful transfer is completed.

Synchronizing master data is also essential for data consistency: the test bed and test management system must always use the same master data. These requirements also apply to other interfaces such as data transfer between the test bed and other connected systems, e.g. for the development of test sequences and methods, but also for setups and settings, operating and setup personnel for documentation, e.g. according to ISO 17025, and for resources.

It is extremely important that consistent data is available to the test bed and test management. Only if both use the same data with the same meaning it is ensured that orders can be defined and executed correctly. For this purpose, lists and enumerations can ideally be better used in both systems instead of free text fields, which can then be synchronized between the systems.

Where the master data is managed also depends on its type: resources, personnel and equipment can be managed very well in the test management system, where they can be reconciled with other higher-level systems such as ERP if required. Master data can also be transferred to the test bed as part of the test order. Test sequences, test methods and setup information are often held as proprietary formats in the test bed automation.

Therefore, this information is better kept there. The test bed automation system tells the test management system which elements are available and which "capabilities" are available. The main element is always the test order. Which sequences and resources were used to process a test order should therefore always be stored and passed on considering the context of the test order.

Especially when many test beds or long-running tests have a significant share, the interface for the transmission of status information from the test beds becomes very important. This includes information such as error messages, but especially the operating status of the test bed and the execution status of the tests. In addition, parameter values can also be useful for status monitoring. And in the case of endurance tests, regular data records are added as intermediate statuses that can be used for trend evaluation.

A good knowledge of the current state of the test bed and the test sequence is of great importance: On the one hand, central monitoring allows less staff presence to be required at the test bed during long-running tests. This means that less staff time is required and employees can concentrate on important other tasks.

On the other hand, the early detection of faults and abnormalities on the test bed allows test beds and devices under test to be protected from destruction in the event of slowly developing material failure. Failed tests can then be detected at an early stage, aborted and restarted. The test bed is thus utilized much more efficiently. The automation of monitoring by means of rules (or even AI methods) also enables predictive maintenance of the test beds.

The interface between the test beds and the test management system can be implemented in different ways. For example, the test beds can use the web service API of the test management system to make status updates there. However, this requires that all test beds implement this. Therefore, an industrial standard interface such as OPC-UA, which is widely used, is more suitable. OPC-UA can be used primarily for a process data server, which can run either at each test bed or also centralized as middleware.

In any case, certain risks must be taken into account when directly accessing the test bed during test execution: On the one hand, test execution can potentially be disrupted if, for example, write access is also permitted instead of exclusively read access. On the other hand, an inadequately secured process server carries the risk of data being viewed by unauthorized persons. Concepts and architectures must therefore be selected for the interface for transmitting status information that take both efficient operation and security into account as important aspects.

In addition to interfaces resulting directly from the main process of testing, there are other interfaces resulting from the accompanying processes. One of these interfaces is used to provide test procedures - commonly referred to as test sequences.

Test sequences are often developed and released centrally. They consist of the test steps that are to be executed in automatic sequence. Before execution, they are often parameterized with variables. Initial basic settings and parallel background monitoring can also be part of the sequences. Sometimes the test sequences are also provided with pre- and post-test steps, e.g. for calibrations or checklists with user instructions for set-up and dismantling.

For this purpose, a central development and release process makes sense from both an efficiency and a quality point of view. This is the only way to establish a good quality assurance process and achieve standardization, which ultimately leads to the desired gain in efficiency.

When centrally developing and managing the test procedures in the test management system, existing process and data management mechanisms can often already be used. Likewise, the sequences and configuration can already be selected during test planning in the test management system, which leads to considerable time savings on the test bed.

For the interface to the test bed, care must be taken to ensure that no conversion of sequences and parameterizations has to take place. Instead, the sequences should be available in the native format of the test bed automation. Alternatively, at least standardized formats should be used which are converted into the necessary formats.

The transfer of the released sequences can be done via network drives. However, since accompanying information is often also required, such as the type of test, other ways are usually more suitable. Either the sequence can be attached to the test order as a file, or - if sequences are to be managed independently of the orders for organizational reasons - concepts of version management systems from software development are used. This allows versions of test sequences to be managed centrally and retrieved by the test bed for local storage.

Analysis procedures can be handled in a similar way to test sequences: On the one hand, derived variables are often calculated online from the measured values at the test beds in order to be able to display and record the relevant parameters. On the other hand, initial evaluations of the data are already carried out at the test beds, e.g. to validate that the test was carried out correctly before the device under test is removed from the test bed.

It is important here that the information from the test definition can be integrated into the evaluation as parameterization. The corresponding scripts for performing the analyses are very similar to the test sequences: they are files that should undergo a release process for quality assurance as part of the development. This means that the same procedures for management and development come into play as for test sequences: the test management can support a release-based development cycle and provides the configuration and parameterization of the analyses for the individual tests.

It makes perfect sense to perform analyses decentrally, because this reduces the load on the test bed and frees it up for further tests at an early stage. Especially when it comes to long-running and computationally intensive evaluations, a decentralized analysis makes sense. However, there are also reasons for carrying out the analysis directly at the test bed, because then it is possible to analyze the data promptly and, if necessary, to repeat a test immediately, as long as the test bed and the device under test are still available. Also, the test bed operator can already evaluate and comment on the information of the analyses before all data are transmitted collectively to the test data management.

Since there is a certain similarity between the processing of test sequences and analyses, the distribution of sequences and analyses can be done via similar procedures. For the execution of the analyses, it is necessary to include appropriate analysis tools, which requires internal interfaces within the automation that are adapted to the tools used.

This interface is similar to the interface to test sequences and analysis scripts. The software on the test beds consists of several components: Operating system, runtime engines, utilities such as firewalls, antivirus software, PDF viewers, etc., as well as the actual test bed automation software and analysis tools, if any. These must be transferred to the test beds and installed there. They must then be maintained and kept up to date.

With a large number of test beds, this can be a major challenge:

• Different versions need to be managed.
• Efficient planning of updates in the context of test bed utilization and availability is an important aspect.
• For quality assurance purposes, it must be documented which software versions were installed on a test bed at which point in time.
• Comparability of results must be ensured, and errors can creep into manual update processes.

As with test sequences and analysis scripts, a software version management tool is a good choice.

Software update scheduling can be integrated with the test management system to align scheduling with test bed utilization. Full integration of test management and version management also makes it possible to detect when tests are being run on test beds with outdated versions.

The ideal solution is to use update services on the test beds that search for new versions, download them and prepare them for installation. Then the installation can be triggered by IT during idle phases through remote access or performed directly by the operator at the test bed. But at the very least, centralized management and distribution should be in place.

For integrated systems, system failures are a particular challenge. Especially for the test bed, for example, the network connection between the test bed and the higher-level test process management system is of elementary importance. If the network connection to the higher-level system is interrupted, the test bed automation system must draw attention to this at an early stage via an alarm so that the IT and system administrators can fix the existing problems as quickly as possible. For this case, the option of local temporary storage of all already available test orders directly on the test bed and the use of a simple local test management is recommended. From this, tests can be temporarily selected for execution.

In order to be able to continue working smoothly even if connection problems persist for a longer period of time, the local test management should also provide the option of being able to create and manage test orders locally. This means that a test definition can also be created locally on the test bed as a complete data set if necessary.

The local test management system is then well equipped for the subsequent transfer of test order data and test result data to the higher-level system. "Test result data" here refers to the entirety of all information occurring at the test bed. In addition to the measurement results of all connected systems, this also includes the photos, comments and reports stored for a test order, which may already have been generated by initial analyses at the test bed.

However, the higher-level system must also offer the necessary interface functions for the transfer. A flexible system strategy that takes open interfaces into account is the best choice for these cases.

For serious failures of the test bed automation system, such as a failure of a server computer, a redundant, identically designed system can also provide protection.