ATS Deployment Guide

Introduction

This document describes the procedure to setup and run the automatic test system. This assumes that the machine is freshly provisioned.

Reference documents

Document

code

Version

1

Deployment document

3151_MCS_0036

1.0

Hardware configuration

In this section the needed hardware and its configuration is explained.

Development PC

This is a Windows 10 machine where LabVIEW simulators and tools will run. It is currently a Virtual Machine located on a hypervisor under the url tma-windows.ls.lsst.org. It uses RDP for remote access and is under the LDAP domain. It can be accessed on the lsst-wap network or by using the openconnect vpn. Access can be granted by filing an IHS ticket with IT. Also, a specific tool to manage the model in the Speedgoat is running here.

Labview

  • You need your NI account to be granted access by IT to activate a license on the LabVIEW License Server located on nlm-labview-1.lsst.org

Tools needed:

  • LabVIEW 2020 - installer zips located on Pavo or on NI’s website with NI SSP enabled account

  • LabVIEW License manager - Download on NI’s website

  • JKI LabVIEW VI Package Manager - JKI account needed - Free edition is good enough

  • NI MAX - included with LabVIEW

  • LabVIEW packages listed in the table in section 4 of 3151_MCS_0036 - All of the packages that the installer comes with except for NI statemachine which is now a VIPM package.

  • VI packages listed in section 4 of the deployment document

  • QMT database MySQL can be downloaded from the source repo.

  • Tekniker made VIs - lsst-ts/ts_tma_vipm_dependency private repo

Speedgoat

The Speedgoat is used to simulate the main axis behavior in real time.

  • Speedgoat

    • Options:

      • CPUCorei74200 (ItemID 109211)

      • SSD500GB (ItemID 109048)

  • Input/Output modules: - Ethercat Slaves, IO750 (x2) (ItemID 2B7506) - IO 306

Software configuration will be downloaded using Tekniker made tool.

Tools needed:

PILZ CPU

This will be used to simulate and test safety software

  • PIlz PSSu 4000 ref 314070

  • PSSu E F 4DI-T

  • PSSu E F 4DO 0.5-T

The configuration of hardware is part of the project where the code is included, some configuration will be explained in section 4.

Software deployment

Each machine has different software running on them. In the following section, the software is installed on the computers.

TMA Windows

In the Windows Machine some simulators and some tools are running. Start installing the Force EtherCAT Variables installer, that will install the LabVIEW runtime needed in many other tools and simulators.

Force EtherCAT Variables

This tool allows writing data to EtherCAT variables to other simulators using a TCP based custom protocol. The value written using this tool will overwrite any set value, so any slave value will be overwritten with the written value. The source code and more documentation about configuration can be found in forceethercatvars.

Follow next steps to deploy this software:

  1. Clone the repository forceethercatvars.

  2. Open the project ForceEtherCATVars.lvproj

  3. Go to “Build Specifications” and right click in “ForceIOs” to select “Build”

  4. Go to “Build Specifications” and right click in “ForceEtherCatVars Installer” to select “Build”

  5. When compilation is finished, open location and copy the “Volume” folder to Windows Machine

  6. Install the tool using the “install.exe”

  7. Run ForceIOs.exe.

Read/Write Network Shared Variables Tool

This tool allows reading and writing data from network shared variables to other simulators and uses a TCP based custom protocol. The source code and more documentation about configuration can be found in readvariables(gitlab).

Follow next steps to deploy this software:

  1. Clone the repository https://gitlab.tekniker.es/aut/projects/3151-LSST/hil/readvariables

  2. Open the project ReadVariables.lvproj

  3. Go to “Build Specifications” and right click in “Executable” to select “Build”

  4. When build finishes go to build folder and copy all files and folder

  5. Paste compilation files to desired destination in Windows Machine

  6. Open the “data” folder and open “WriteReadVarConfig.xml”.

  7. Change the path of the field TCP_configuration_file to point to TCP_ServerConfig.xml file in the same data folder.

  8. Run ReadWriteNSVs.exe

This tool is used for reading the variables from 3 different hosts the configuration for each of the instances can be found here: https://gitlab.tekniker.es/aut/projects/3151-LSST/hil/readvariables/-/tree/develop/Configuration

  • ReadWriteAxesPXI_NSVs: the configuration for the instance that reads/writes the variables from the AxesPXI.

  • ReadWriteTMAPXI_NSVs: the configuration for the instance that reads/writes the variables from the TMA_PXI.

  • ReadWriteLocal_NSVs: the configuration for the instance that reads/writes the variables from the WindowMachine.

BoschPowerSupplySimulator

This is a simulator for the bosch power supply, this simulator manages the digital inputs that tell the TMA PXI the status of the power supply. The source code and more documentation about configuration can be found in boschpowersupplysimulator(gitlab)

Follow next steps to deploy this software:

  1. Clone the repository https://gitlab.tekniker.es/aut/projects/3151-LSST/hil/boschpowersupply/boschpowersupplysimulator

  2. Open the project BoschPowerSupplySimulator.lvproj

  3. Go to “Build Specifications” and right click in “Executable” to select “Build”

  4. When build finishes go to build folder and copy all files and folder

  5. Paste compilation files to desired destination in Windows Machine

  6. Run BoschPowerSupplySimulator.exe

motorThermalModelSimulator

This is a simulator for the thermal behavior of the phase motors, this simulator manages the analog inputs that tell the TMA PXI the temperatures of the motors and uses this values to control the output signal of the valve to manage the temperature of them. The source code and more documentation about configuration can be found in motorthermalmodelsimulator(gitlab).

Follow next steps to deploy this software:

  1. Clone the repository in the link above https://gitlab.tekniker.es/aut/projects/3151-LSST/hil/motorthermalmodel/motorthermalmodelsimulator

  2. Open the project motorThermalModelSimulator.lvproj

  3. Go to “Build Specifications” and right click in “Executable” to select “Build”

  4. When build finishes go to build folder and copy all files and folder

  5. Paste compilation files to desired destination in Windows Machine

  6. Run motorThermalModelSimulator.exe

PhasePowerSupplySimulator

This is a simulator for the phase power supply, this simulator manages the analog inputs that tell the TMA PXI the status of the power supply. The source code and more documentation about configuration can be found in phasepowersupplysimulator(gitlab).

Follow next steps to deploy this software:

  1. If the installer or executable is available continue to step 6

  2. Clone the repository in the link above

  3. Open the project PhasePowerSupplySimulator.lvproj

  4. Go to “Build Specifications” and right click in “Executable” to select “Build”

  5. When build finishes go to build folder and copy all files and folder

  6. Paste compilation files to desired destination in Windows Machine

  7. Run PhasePowerSupplySimulator.exe

Simulate limits

This software allows to simulate the behavior of some subsystem limit switches. Those limits could be part of safety system or EtherCAT distributed IOs. The source code and more documentation about configuration can be found in simulatelimits(gitlab).

Follow next steps to deploy this software:

  1. If the installer or executable is available continue to step 6

  2. Clone the repository in the link above

  3. Open the project SimulateLimits.lvproj

  4. Go to “Build Specifications” and right click in “SimulateLimits” to select “Build”

  5. When build finishes go to build folder and copy all files and folder

  6. Paste compiled files to desired destination in the Windows Machine

  7. Open the “data” folder and open “GeneralConfiguration.xml”

  8. Change the first path of the field TCP_senders_configuration_Path to point to ForceECATVars_TCP_SenderConfig.xml file in the same data folder.

  9. Change dim=’[X]’ to dim=’[1]’ for “TCP_senders_configuration_Path” and for “LimitsDefinition” tags. We are only using the first configured limit because you need the safety full simulator with PILZ hardware to use other limits, when you get this hardware (perhaps you have one on the submit) we can download code to it and use those other limits.

  10. Run SimulateLimits.exe

cabinetTemperatureControllerSimulator

This is a simulator for the temperature controller of the cabinets, this simulator contains the simulator of the different temperature controllers available all over the telescope. The source code and more documentation about configuration can be found in cabinet-az0001(gitlab).

Follow next steps to deploy this software:

  1. If the installer or executable is available continue to step 6

  2. Clone the repository in the link above

  3. Open the project cabinetTemperatureControllerSimulator.lvproj

  4. Go to “Build Specifications” and right click in “Executable” to select “Build”

  5. When build finishes go to build folder and copy all files and folder

  6. Paste compilation files to desired destination in Windows Machine

  7. Run cabinetTemperatureControllerSimulator.exe

The cabinets included in this simulator are:

  • TMA_AX_DZ_CBT_0001 (Phase Main Power Cabinet)

  • TMA_AZ_CS_CBT_0001 (TEK Mount Control System cabinet - MCS)

  • TMA_AZ_PD_CBT_0001 (Azimuth Power Distribution)

  • TMA_AZ_PD_TRM_0001 (Isolation transformer)

  • TMA_EL_PD_CBT_0001 (Elevation Power Distribution 1)

  • TMA_EL_PD_CBT_0002 (Elevation Power Distribution 2)

extensionSimulatorForDP

This is a simulator for the extensions of the deployable platforms, this simulator manages the digital inputs that tell the Safety system the status of the extensions of the deployable platforms. The source code and more documentation about configuration can be found in dpextensionssimulator(gitlab).

Follow next steps to deploy this software:

  1. If the installer or executable is available continue to step 6

  2. Clone the repository in the link above

  3. Open the project DPextensionsSimulator.lvproj

  4. Go to “Build Specifications” and right click in “Executable” to select “Build”

  5. When build finishes go to build folder and copy all files and folder

  6. Paste compilation files to desired destination in Windows Machine

  7. Run extensionSimulatorForDP.exe

OilSupplySystemSimulator

This is a simulator for the Oil Supply System (OSS), this simulator contains a modbus server that connects to the TMA PXI to transmit the status of the OSS. The source code and more documentation about configuration can be found in oilsupplysystemsimulator(gitlab).

Follow next steps to deploy this software:

  1. If the installer or executable is available continue to step 6

  2. Clone the repository in the link above

  3. Open the project OilSupplySystemSimulator.lvproj

  4. Go to “Build Specifications” and right click in “Executable” to select “Build”

  5. When build finishes go to build folder and copy all files and folder

  6. Paste compilation files to desired destination in Windows Machine

  7. Run OilSupplySystemSimulator.exe

Deploy Speedgoat

This provides the motion model for the TMA using specialized hardware. The Speedgoat Manager will handle loading the model and managing the configurations. Before starting to deploy the model, make sure that the SpeedgoatManager is running.

  1. Install matlab 2020a with the following dependencies

  • Simulink

  • Simulink Real Time

  • Simulink coder

  • Matlab Coder

  1. Install Speedgoat IO for Matlab 2020a

  2. Run the slrtexplorer command in matlab

  3. Configure it to look for the target’s ip address as 192.168.17.1

  4. Clone the model repository

  5. Build/deploy and run the model

SpeedgoatManager

This is a simulator tool used for the robot framework tests to connect to the Speedgoat. The source code and more documentation about configuration can be found in https://gitlab.tekniker.es/aut/projects/3151-LSST/hil/speedgoat

Follow next steps to deploy this software:

  1. Get the latest version of the compiled code from here: https://gitlab.tekniker.es/aut/projects/3151-LSST/hil/speedgoat/speedgoatmanagerbinaries

  2. Change the ip address setting to become the windows machine’s ip address

  3. Paste it to the windows machine

TMA Centos

In the Linux Machine the secondary axis simulators and the robot framework tests are running. This is a Virtual Machine running on a hypervisor that is located under tma-centos.ls.lsst.org. It can be accessed either on the lsst-wap network or by using the anyconnect vpn. Access can be granted by filing an IHS ticket with Vera C. Rubin Observatory IT.

secondaryAxisSil

This is a simulator for the secondary axes (bosch axes), this simulator contains a modbus server that connects to the TMA PXI to transmit the status of each of the axes. The source code and more documentation about configuration can be found in https://gitlab.tekniker.es/aut/projects/3151-LSST/hil/secondaryaxis/secondaryaxissil

Because of the use of certain internal libaries in the source code, download the compiled binaries from https://gitlab.tekniker.es/aut/projects/3151-LSST/hil/secondaryaxis/secondaryaxissilbinaries

Follow the steps defined in the secondaryAxisSilREADME.

robotFramework

This refers to the automatic test framework the installation steps to setup the environment for robot framework is explained here:

The source code and more documentation can be found in https://gitlab.tekniker.es/aut/projects/3151-LSST/test/robotframework

Hostnames for the ATS

Name

Host Name/IP Address

TMA PXI

ats-tma-pxi01.ls.lsst.org (139.229.145.241)

AXES PXI

ats-tma-axes-pxi.ls.lsst.org (139.229.145.242)

AUX PXI

139.229.145.238

PILZ

tma-windows.ls.lsst.org (139.229.145.101)

HMI/MCC

tma-centos.ls.lsst.org (139.229.145.102)

HMI

The HMI is installed as a LabVIEW runtime executable on the TMA CentOS machine. The operation manager is installed as an RPM package which leverages systemd service architecture.

TMA PXI

The PXI is a model 1086.

Use NI-MAX to install NI Linux RT system image along with Variable Engine and Ethercat driver.

Then create configuration directories and files using this document.

mkdir -p /c/Configuration/CAR_TCP /c/Configuration/DiscreteStateReporting /c/Configuration/EIB /c/Configuration/Safety /c/Configuration/TekNSVs /c/Configuration/axisManagementComm

Find the files under the ESIfiles directory of the PXIController repo. Most of the folders match the names on the cRIO. Some of the files have the suffix ForATS and those are the ones that you want to copy.

This is the PXI where the control code for all subsystems is running. To be able to configure the TMA PXI, the development PC should be configured as shown in the deployment document.

  1. Download the PXI repository: https://gitlab.tekniker.es/aut/projects/3151-LSST/LabVIEWCode/PXIController

  2. Open the LSST_MainControllerPXIATS.lvproj under the ATS folder.

  3. Ensure that in the project properties the Conditional Disable Symbol “HIL” is set to “True”

  1. Right click in the project an select properties

../../_images/TMAPXIpic1.png
  1. In the opened window go to Conditional Disable Symbols page and set the value for HIL symbol to “True”.

../../_images/TMAPXIpic2.png
  1. Update target address to use ats-tma-pxi01.ls.lsst.org

  2. Save project.

  3. Save the project.

  4. Close the project.

  5. Under tools, clear build cache

  6. Close labview

  7. Open labview

  8. Open project.

  9. Open main.vi

  10. Close main.vi

  11. Configure ethercat project to match hardware connection order and change ID to 0.

  12. Build target

  13. Deploy target

AXES PXI

The PXI is a model 1086.

Start by adding the necessary configuration files from ESIfiles using this document.

mkdir -p /c/Configuration/axisManagementComm

This is the PXI where the control code for the main axes is running. To be able to configure the AXES PXI, the development PC should be configured as shown in the deployment document

  1. Download the PXI repository: https://gitlab.tekniker.es/aut/projects/3151-LSST/LabVIEWCode/PXIController

  2. Open the LSST_MainAxesPXIATS.lvproj under the ATS.

  3. Ensure that in the project properties the Conditional Disable Symbol “HIL” is set to “True”

  1. Right click in the Axes PXI an select properties

../../_images/TMAPXIpic1.png
  1. In the opened window go to Conditional Disable Symbols page and set the value for HIL symbol to “True”.

../../_images/TMAPXIpic2.png
  1. Update project target to ats-tma-axes-pxi.ls.lsst.org

  2. Save project.

  3. Close project.

  4. Clean build cache

  5. Close LabVIEW

  6. Open LabVIEW

  7. Open project

  8. Open main.vi

  9. Close main.vi

  10. Configure ethercat with ethercat project to match connection order

  11. Build target

  12. Deploy to target

The AXES cRIO runs FPGA code that needs to be deployed. Use the NI Package Manager to install the FPGA package and its dependencies. Otherwise find a FPGA compile server that’s located on the network.

  1. Open the AXES (not for ATS) project.

  2. Change the AXES target IP address to the AXES ATS IP address.

  3. Open the FPGA target under the cRIO folder.

  4. Build the FPGA target locally or with a FPGA compile server.

  5. Download to the target.

Note

The simulink model needs to be deployed on the speedgoat before the fpga code will work.

AUX PXI

The PXI is a virtual machine running NI RT linux.

Start by setting up the configuration directory using this document. You can find the files in the ESIFiles directory inside of the PXIController repo.

  1. Open AuxMainControllerATS under ATS folder.

  2. Save project.

  3. Close project

  4. Clear cache

  5. Close labview

  6. open labview

  7. Open project

  8. Open main.vi

  9. Close main.vi

  10. Build target

  11. Deploy network variables

  12. Deploy target

Safety code deployment

The code that runs on the PILZ controller to simulate the behavior of the TMA Interlock System. The source code and more documentation about configuration can be found in testdualmodbus(gitlab)

  1. Open the “TestDualModbus” project with PAS4000 version 1.18.0

  2. Activate the “TestDualModbus”

../../_images/PASS4000activateProject.png
  1. Open the online network editor

../../_images/PASS4000onlineNetworkEditor.png
  1. Scan project to scan the network to verify that the PILZ CPU is connected

../../_images/PASS4000scan.png
  1. Close the online network editor

  2. Download the project

  1. Open the Project downloader:

../../_images/PASS4000downloadCode.png

If asked to build changes say YES

../../_images/PASS4000buildChanges.png
  1. Start download:

../../_images/PASS4000startDownload.png
  1. Confirm download:

../../_images/PASS4000confirmDownload.png
  1. Download completed:

../../_images/PASS4000downloadCompleted.png
  1. Logout:

../../_images/PASS4000logout.jpg
  1. Close the PAS4000