[Jon, Tyler, Aiden, Luke]

On Friday we completed assembly of the new stainless steel-topped benches in 1129. We then moved the clean and bake equipment from 1119 to its new larger space in 1129. This included the HEPA flow bench, ultrasonic washer, deionized water drum, nitrogen tanks, and forced-convection oven. The oven was re-anchored to the wall with earthquake restraints in its new location.

The power cords still need to be permanently routed, but the new clean and bake lab is otherwise ready for use.

Update: I have completed permanent routing of the electrical cables. I also ran the ultrasonic washer's drain line to the sink drain (the current hose does reach). The new clean and bake lab is now fully operational.

Cleaned and baked the Cal Tech FROSTI legs and showed Luke the procedure on how to clean parts.

[Luke, Luis]

On Friday, 11/1/24, the MR was unpackaged and cleaned under the flow bench. We noticed an ink stain on one of the corners of the material. After wiping it down with IPA and Vector Alpha wipes, the stain was removed. However, the material showed some wear.

After cleaning, we placed it in the bake station and applied a stainless steel baking protocol. On Tuesday, 11/4/24, the material was removed from the furnace and packaged in a static shielding bag. The MR was wrapped in Vector Alpha wipes.

Today I fixed the final bit related to the nitrogen gas tank, which is to apply sealing tape to M-NPT connector of the hose to prevent leakage (file: AirGunSealed.jpg)
After application of the tape, no audible leak can be heard from connection between the hose and the air gun.

The general operating procedure for the gas tank is as following:

1. Turn the regulator (blue handle) anti-clockwise still it's loose
2. Turn the valve on nitrogen as tank anti-clockwise, immediately the RHS meter of the regulator would jump to approx 2000 psi. This is the standard pressure for high pressure gas tank
3. Turn the regulator clock-wise slowly until the pressure one the LHS meter face reads approx 60 psi. This is sufficient for drying parts with. At this point, the flow pressure still should register zero
4. Press the trigger on the air gun, a high pressure air flow should come out and the flow meter should increase
5. When finished, close the gas tank valve, turn the regulator anti-clockwise, then press the air gun trigger to release gas left in the hose/gun

The new FLIR A70 infrared camera has arrived. Tyler and I unpacked it in the lab yesterday. In less than an hour, we succeeded in powering it on and connecting it to the lab network. We have assigned it the static IP address 192.168.1.6.

Online Configuration Portal

The FLIR camera can be configured, as well as stream live data, through a web browser interface. It can be accessed from any workstation on the lab network by navigating in the browser to http://192.168.1.6. The login credentials are stored here (log in with your LIGO.ORG credentials).

Next Steps

The next step is to install FLIR's Python API for controlling and reading out the camera on chimay. The API comes with demo codes which we can use to test the basic connectivity and which will serve as a reference for developing our own Python interface over the summer.

Summary

I have installed the requisite software on chimay for interfacing the FLIR A70 camera in Python. There are two packages required from FLIR:

• Spinnaker SDK, which provides the low-level camera drivers and a C/C++ API.
• PySpin, a wrapper of the Spinnaker library which provides the Python API.

These installations did not work out-of-the-box for Debian 11 (only Ubuntu is officially supported). I had to make several modifications which are documented below for future reference.

This setup has not yet been tested with the camera connected to chimay.

Documentation and Demo Codes

The PySpin package comes with a number of Python demo codes and a complete API reference. These can be found on chimay at the following locations.

• Example codes: /opt/spinnaker/python/Examples/Python3/
• Python API reference manual: /opt/spinnaker/python/docs/PySpinDoc.pdf

Installing Spinnaker SDK

Below were the steps required to install Spinnaker on chimay (Debian 11).

1. Download the Spinnaker binaries (AMD64 architecture) and copy the tarball to, e.g., /home/controls on chimay.
   
   
2. Unpack the tarball contents and enter the new directory:
   $ tar -xf spinnaker-2.6.0.160-Ubuntu20.04-amd64-pkg.tar.gz
$ cd spinnaker-2.6.0.160-amd64
   
   
3. Next, install the dependencies (on chimay, these were all already installed):
   $ sudo apt-get install libavcodec58 libavformat58 \
libswscale5 libswresample3 libavutil56 libusb-1.0-0 \
libpcre2-16-0 libdouble-conversion3 libxcb-xinput0 \
libxcb-xinerama0
 
4. There is one additional dependency, qt5-default, which is obsolete in Debian and no longer available via the package manager (that is, its functionality was absorbed into other Qt packages). I was able to find a workaround based on these instructions.
   
   
   1. Install all the dependencies of qt5-default:
      $ sudo apt-get install qtbase5-dev qtchooser qt5-qmake qtbase5-dev-tools
 
   2. Manually remove the qt5-default dependency from the Spinnaker package.
      
      Unpack the spinview-qt_2.6.0.160_amd64.deb package:
      $ mkdir tmp
$ cd tmp
$ ar -x ../spinview-qt_2.6.0.160_amd64.deb
$ tar xf control.tar.xz

Open the file control in a text editor and delete the qt5-default dependency from the Depends list.
      
      Then repackage the contents:
      $ tar cfJ control.tar.xz control
$ ar rcs ../spinview-qt_2.6.0.160_amd64.deb debian-binary control.tar.xz data.tar.xz
$ cd ..
$ rm -rf tmp

This overwrites the original package with a version no longer containing the qt5-default dependency.
      
      
5. Now proceed with running the install script:
   $ sudo sh install_spinnaker.sh

This will install the Spinnaker library at /opt/spinnaker. Spinnaker also provides a standalone GUI application, SpinView, which can be executed from the terminal (from any directory) via the command spinview.

Installing PySpin

The main challenge with installing PySpin was that it is currently only supported for Python <=3.8. The system installation on Debian 11 is Python 3.9 and 3.8 is not available within the package manager. Following these instructions, I manually installed a second version of Python (3.8) on chimay, in a way that should not interfere with the system installation.

The Python 3.8 executable is in the system path and can be run only via the command python3.8. It is not symlinked to python or to python3. Those remain linked to the preexisting Python 3.9.

After installing Python 3.8, I proceeded with the installation as follows:

1. Download the PySpin package (x86_64 architecture) and copy the tarball to, e.g., /home/controls on chimay.
   
   
2. Unpack the tarball contents and into a new directory:
   $ mkdir python
$ mv spinnaker_python-2.6.0.160-Ubuntu20.04-cp38-cp38-linux_x86_64.tar.gz python
$ tar xf spinnaker_python-2.6.0.160-Ubuntu20.04-cp38-cp38-linux_x86_64.tar.gz


3. Move the new directory into the Spinnaker installation directory:
   $ sudo mv python /opt/spinnaker
$ cd /opt/spinnaker/python
 
4. Install the dependencies:
   $ sudo python3.8 -m pip install --upgrade numpy matplotlib
   
   
5. Finally, install PySpin itself:
   $ sudo python3.8 -m pip install spinnaker_python-2.6.0.160-cp38-cp38-linux_x86_64.whl

If this succeeded, you should now be able to enter import the package PySpin as

$ python3.8
>>> import PySpin

without error.

Today I tested the Spinnaker/PySpin software installations (detailed in ELOG #4) with the FLIR camera connected to chimay. It works!

Example codes

I was able to run several of the PySpin example codes. In particular, there is one which connects to the camera and streams live data to a pop-up Matplotlib window that looks very useful. It is called AcquireAndDisplay.py.

When running these, it is important to keep in mind that PySpin requires Python 3.8, which is not the default system version on chimay. So to run AcquireAndDisplay.py, for example, you must explicitly call the correct version of Python:

$ python3.8 AcquireAndDisplay.py

The standard python and python3 aliases are still linked to the system version (3.9), so calling these will result in a PySpin import error.

Git repository

I have set up a git repo for our FLIR camera control code. I have populated it with an Examples directory which contains the PySpin Reference Manual as well as all the example codes (see the README). There is a local copy of this repo on chimay at /home/controls/FLIR.

Other FLIR streaming software

In addition to the PySpin demos, there are several fully developed applications provided by FLIR. While we do not plan to use these long term, they may be very useful for debugging and cross-validation of our Python interface during development:

• Browser interface: From any web browser on the local lab network, navigate to http://192.168.1.6 and log in (credentials here). This interface supports live data streaming as well as full control of the camera settings.
  
  
• SpinView: A standalone application provided as part of the Spinnaker SDK. It supports streaming live camera data as well as saving images and videos. It can be launched from the terminal on chimay via the command: $ spinview
  
  
• Research Studio: This is FLIR's proprietary software, for which we have a one-year license. It can be launched from the terminal on chimay via the command: $ FLIRResearchStudio

Permanent cabling

Since everything appears to be working, I ran a permanent Cat 6 cable from the lab switch to the camera's power+I/O adapter. The adapter is plugged into a UPS-protected power strip overhanging the optical table, as pictured below. To prevent the adapter from unplugging itself under its own weight, I attached a zip tie around the adapter to hold it securely in place.

Screen Setup

The screen was set up by clamping it between two rectangular posts on each side. First, two posts were set up with 22 in. between them (thus allowing the screen to span a total distance of up to 24 in. when clamped down). To best stabilize the screen and to allow for it to be "pulled" taut by exploiting the give in the L-bracket, the L-bracket was bolted on the outside of the post, along the same axis as the screen itself.

On first attempt, the screen was too thin to be fully clamped between the posts. In order to have it fit snugly, sections of heat shrink tubing was used as a shim at the points where the posts were clamped together. The tubing was slid into the track of one optical post at the desired points. In order to accommodate the shim, the two posts had to be held together, ideally clamped, with the screen and shim in place. Then, the post clamps could be slid into the tracks of the post, moved to the optimal location, and tightened down. This required at least three people: two to tighten one side while the third holds up the screen on the other side. The screen was placed ~1/8" from the edge of the posts and flush with the top.

Camera Stage Setup

Once the screen was in place, the camera stage was set up by placing the XY-Translational with Rotation stage on four 3" optical posts. Then, the z-axis stage was placed in the center of that, with another 3" optical post on top, which was then topped with the camera. This was set and clamped down ~22.5" from the screen. This was about 1" closer than expected based on our theoretical models.

Fine Alignment

We used the visible camera to fine align the screen and to test the setup. Notably, the visible camera is placed below the infrared and thus requires a calibration in order to ensure the two are aligned on the computer image. This can be set by hand using the FLIR proprietary software (FLIR CamWeb) and adjusting the "MSX alignment". The image mode "Thermal MSX" allows both the visible and IR camera to be displayed at once and the difference in their positioning can be seen. We found an offset of ~0.5m to be nearly accurate (note: using this method, although you can get more accurate than this, the displayed value only has one significant figure).

In order to align the camera, we first used the exposed top edge to judge whether the camera was appropriately centered on the screen. We set the rotation as close as possible to being in line by eye, then adjusted the y-axis until the gap on both corners was a similar size, thus indicating that the rotation and y-position were correctly set. Rotationally, the camera required only a refinement of -1/2 degree. The y-axis is set at 1.25. Then, the camera was pulled as far from the screen as possible using the x-axis to allow the screen to be easily centered using the z-axis. Once the outlined test mass was centered, the x-axis was used to bring the camera close until the screen just barely filled the field of view. The x-axis is at 2.25. The z axis is set to it full dynamic range at 10. Unfortunately, the camera is still slightly too tall for the screen, likely requiring the purchase of a new optical post about 0.5in shorter the current one. This interchange will likely require a new fine alignment after.

Basic Imaging Tests

The camera was also focused on the screen based on the manufacturer's printed distance on the camera itself (using 22.5", or 0.572 m). Using the FLIR proprietary software, the camera appears to be in focus in IR (a hand was used as a good focusing tool for this). Additionally, the camera does pick up heat on the other side of the screen. A hand can be lightly seen warming the screen, as can a soldering iron tip. This was a very imprecise visual tool, but does indicate that the camera and screen are working roughly as expected.

Next Steps

A new optical post that is ~2.5" tall should be ordered to replace the one under the camera currently. The heating system also needs to be ordered and set up. Currently we are debating between a parabolic reflector with a hole in the back, and one without, as each would require a different mounting mechanism for the cartridge heater.

There were no issues with the physical replacement of the post, except that the fork clamp on the post needs to be on one of the perpendicular, not diagonal, axes in order to be secure. I chose the front axis (towards the screen) as before in order to easily access the alignment knobs.

To align, I followed the same process as last time except for a more purposeful original rough alignment. For alignment purposes, the visual camera was used. For the first rough alignment, I pulled the camera as far back as possible on the x axis (with the z axis roughly centered), then moved the entire stage setup back until I could just see both the top and bottom edge of the screen. Then, I set the z-axis to an extreme in order to use the edge of the screen to align the rotational and y-axis pieces for the fine alignment.

For the fine alignment, starting with the z and x axis at extremes, I began by aligning the rotational axis. To do this, I used the gaps between the top of the screen and the camera window on the far left and right of the image. When these gaps were equal I knew the rotation was adequately set. Then, I set the y-axis so that the pattern was centered. If the gaps were no longer even, I redid the rotation alignment and ditto with the y axis until both were set. This resulted in a rotation of about two degrees and a y axis at just under 3.5.

To set the z and x axis, I centered the z-axis using the top and bottom of the screen, which should both be visible if the rough alignment was done correctly. Then, I adjusted the x axis by pushing it forward until the top and bottom of the screen were just out of the frame. As with the rotational and y-axis, I iteratively fine tuned the x and z axis until both the image was centered in the z axis and only the screen was in view. This resulted in a z-axis value of just over 5.5 and an x axis value of nearly 1.75.

Pictures are included of all alignment knobs and the new post/stage setup!

Today's CyMAC work:
• Finished assembling external power supply cable for the timing chassis (see attached image)
• Also did some debugging of AA and AI chassis re: power issues. After testing voltages at various test points on the power regulator board (TP1-TP6) in the AA chassis, it seems like it's the source of the problem. Used bench DC power supply to test, running 15V, and the correct voltage is coming in to the board but something significantly smaller is being output. We're also getting unexpected negative signs on voltage at certain test points, with leads in correct positions
• Upon further examination, it looks like the lights on the filter boards are actually turning on (in both the AA and AI chassis), though it's only 2/4 lights on each board and they are very faint
• Corrected DC on/off switch spade lug orientation in AI chassis.
• Tested AI chassis power regulator board to see if the problem was the same, and found again that voltage coming in was correct, and voltage going out was not.
• In case of interest in exact numbers, results were as follows:
  • Voltage difference between TP1(+Vin) and TP6(-Vin) is ~30V
  • Voltage difference between grounded chassis wall and TP1 (+Vin) is ~15V
  • Voltage difference between grounded chassis wall and TP6(-Vin) is ~-15 V
  • voltage difference between TP2(+Vout) and TP5(-Vout) is ~3V
  • Voltage difference between TP3 (gnd) and TP2(+Vout) is -0.065V
  • Voltage difference between TP5(-Vout) and ground is ~3V

• Topmost Sorensen: +24V (reserved for FROSTI)
  • Positive terminal:
  • unconnected for now
  • Negative terminal: [negative terminal is grounded]
  • jumper to Sorensen ground screw
• Sorensen 2nd from the top: +18V
  • Positive terminal:
  • white power cable wire for AA chassis
  • white power cable wire for AI chassis
  • white power cable wire for BI chassis
  • white power cable wire for BO chassis
  • Negative terminal: [negative terminal is grounded]
  • jumper to Sorensen ground screw
  • black power cable wire for AA chassis
  • black power cable wire for AI chassis
  • black power cable wire for BI chassis
  • black power cable wire for BO chassis
• Sorensen 3rd from the top: -18V
  • Positive terminal:[positive terminal is grounded]
  • jumper to Sorensen ground screw
  • green power cable wire for BI chassis
  • green power cable wire for BO chassis
  • Negative terminal:
  • green power cable wire for AA chassis
  • green power cable wire for AI chassis
• Bottommost Sorensen: +6V
  • Positive terminal:
  • white power cabe wire for timing chassis
  • Negative terminal: [negative terminal is grounded]
  • jumper to Sorensen ground screw
  • green power cable wire for timing chassis
  • black power cable wire for timing chassis

[Jon, Shane, Luis]

My repair of the internal ribbon connecting the ADC to the adapter board resolved the timing signal problem. After this repair, we were able to start the front-end IOP model and checked out the RTS diagnostic screens (pictured below). All indicator lights were green except for the DK flag (indicating the DAC outputs are not enabled) and the DAQ flag (indicating that the system is recording data to disk). Those were both as expected, because the DAQD data acquisition service was not set up yet and the DAC outputs are not enabled until at least one user model (which outputs signals to the DAC) is started. I created and installed a simple user model (C1MSC) and confirmed that the DK flag clears once this model starts.

I later attempted to set up the DAQD service, which is needed to save data, but am yet to successfully debug it. I have received some guidance from one of LIGO's CDS experts and will try it at my next opportunity for lab work.

Update: I was able to put the FROSTI power controller on the lab network. It is connected to the switch in the top of the rack and is assigned a static IP address of 192.168.1.12 and an NDS hostname of relay1.

The controller can be remotely accessed through an SSH command line interface as well as an HTML webpage, which can be opened from any web browser on the lab network by navigating to the above IP address (the login credentials are the same as for the workstation computers).

There is also an unofficial Python package for interfacing with the controller: dlipower. We will investigate using this package to interface the controller with soft EPICS channels hosted on the CyMAC. This will allow us to create a custom MEDM screen for controlling the FROSTI heater elements.

Edit: The login credentials were set up to be the same as for the CDS workstations.

Summary

Today I finished implementing an RTS model to read out the integrated FROSTI RTDs (temperature sensors) via the CyMAC. The model is named "MSC" and is located at cymac:/opt/rtcds/usercode/models/c1msc.mdl. We successfully tested it with the heater elements operating in vacuum at low power (12 VDC), finding them to reach an average steady-state temperature of 160 C.

From the cymac host, the MEDM control screen can be accessed with the terminal command "sitemap" (from any directory).

Measurement Technique

Each FROSTI heater element [299] contains an internal two-wire RTD placed near the front emitting surface, which enables the temperature of the blackbody emitter to be directly monitored. From the measured temperature and the emissivity of the uncoated aluminum nitride surface (known to be ~1 in the IR), the radiated source-plane power can also be estimated.

The resistance of each RTD is measured via a ratiometric technique. The RTDs are powered in series with a 1 kΩ reference resistor located inside the readout chassis [305], whose temperature is not changing. The signal is obtained by taking the ratio of the voltage difference across each individual RTD to the voltage difference across the reference resisitor. The advantage of this technique is that the ratio of  the voltage differences is insensitive to changes in the current through the resistors (since they are all in series; see [271] for wiring diagram).

Implementation Detail

The signal flow is shown in Attachment 1. The eight RTD signals enter through ADC channels 0-7, along with the reference resistor signal on channel 8. The first set of filter modules apply a calibration gain to convert the signals from raw ADU counts to units of input-referred voltage. The ratio of each RTD signal to the reference resistor signal is then taken. The second set of filter modules multiply the voltage-difference ratios by the resistance of the reference resistor, 1 kΩ ± 0.01%, to obtain the RTD resistances in physical units of ohms.

Finally, a freeform math module is used to invert the quadratic relation between each RTD's resistance and temperature. The final signals passed to the third set of filter modules are the RTD temperatures in physical units of degrees C. The temperatures of the tungsten RTDs are estimated assuming TCR coefficients of A=0.0030 C^-1 (±10%) and B=1.003E-6 C^-2, which were provided by the manufacturer.

One DAC channel is used to provide the excitation voltage for the RTD measurement, which is visible on the far right of the control screen. At its maximum output voltage of +10 V, the DAC can drive a maximum current of 10 mA.

Finished ADC-DAC loopback testing today (see attached xlsx file or access directly here). All looks well with the first 8 channels, with the gain hovering just under 2. Also edited c1msc model in simulink to add channels 7-15 (the last 8 channels), and changed the rate back to 64K. See image 1 for the updated c1msc model. The last 8 channels are also looking good and show no problems, with slightly more scattering for the gain, but all values very close to 2.

We also installed the new eight-channel Perle IOLAN SDS8 terminal server today. Image attached.

NOTE: When we were installing it, we noticed an energized wire dangling from the 24V power supply. Has since been fixed and put back into place.