Finished ADC-DAC loopback testing today (see attached xlsx file or access directly here). All looks well with the first 8 channels, with the gain hovering just under 2. Also edited c1msc model in simulink to add channels 7-15 (the last 8 channels), and changed the rate back to 64K. See image 1 for the updated c1msc model. The last 8 channels are also looking good and show no problems, with slightly more scattering for the gain, but all values very close to 2.

We also installed the new eight-channel Perle IOLAN SDS8 terminal server today. Image attached.

NOTE: When we were installing it, we noticed an energized wire dangling from the 24V power supply. Has since been fixed and put back into place.

The timing chassis used for the cymac has been shut off due to an unknown issue causing its supplied current to fluctuate. All real-time models will be suspended until a solution is found.

[Ma, Luis, Shane]

Working theory for the timing chassis issues had been that the 1A breaker was tripping and causing the failure of the Valon 5015 and 3010 to output the timing signal correctly. We just tried bypassing the breaker, running 6 V on the benchtop power supply (set the current limit to 1.5A), with the 5010 generating the sine wave to pass to the 3010. All worked correctly, and there were no issues. Square wave outputted by the 3010 was exactly as desired (image attached) at the correct frequency, and this confirms the issue was the breaker, not the valon 5015. Ready to go ahead with ordering a new replacement breaker.

We replaced the 1A breaker in the timing chassis today with a 4A one, and tested that all is working well. The chassis successfully outputted the correct signal (image attached). The real time models have also been restarted and the CyMAC diagnostics screen is showing all green flags. Timing chassis has been closed up and reinstalled in the server rack.

Debugging update for vac system serial communications: we were able to successfully connect the smaller turbo pump to the Varian (manufacturer) software today by plugging the controller db9 directly into spica, and verified that the pump was able to send and receive information. Still no success on communication with the larger pump via python serial code, though. Issue potentially lies in the pinout mapping of the connectors, as the pinouts we'd been using work successfully for the small pump which only requires 3 connections (RXD, TXD, GND), but the large pump may require more. Tried to test this on the big pump by keeping the pinout setup for the small pump, and incrementally adding the remaining 6 connections on a one to one scale (Pin 1 matched to Pin 1, etc) between the controller DB9 and the field DB9 connected to the IOLAN ethernet cable. After each new connection was added, I retried running the serial connection code. None of the additional connections yielded any results, and the large pump remained unable to connect through the code. Next step is further looking into documentation on the pinouts to see if the large pump requires a different connection setup than the small pump.

[Shane, Jon]

Update on the serial interfacing with the vacuum system: both turbo pumps are now able to successfully communicate through the code. The older pump, for unknown reasons, required a newer version of the communication syntax (despite the newer, small pump being able to communicate fine with the old communication format). We have also confirmed that the newer pump works with this new format as well. Next step is interfacing the pressure gauge.

[Shane, Jon]

There are three major issues with the lab's CDS system that should be addressed. They are ordered from the highest priority to the lowest.

• CyMAC has stopped serving the fast channels (65k Hz)
  The channels are still there, and the data from the slow (16 Hz) channels is served correctly, but the fast data is not served. As a result, diagggui can not retrieve any information at the moment.
  Probably the issue is the same as the version conflict between daqd and nds2-server. After installing nds2-server on CyMAC and realizing that the conflict can not be resolved, I reverted the changes so that the CyMAC could be compatible with daqd, but there is a chance that there are still version conflicts that block the fast channels.
  I suspect that we have broken the MSC model when adding the new button. This could explain both the fasr-channels fault and the unexpected crashes.
  
  UPDATE 6/5
  Fixes this by rebuilding the models.
  Note for the future: The c1iop and c1msc models should be rebuilt together, even if no change has been made to c1iop.
• The fast channels were not saved to the frames
  Reviewing the frame files showed that the daqd was only saving the 16 Hz data in the .gwf frame files. I had missed this because I only checked the existence and the integrity of the raw frame files, but never retrieved the high-frequency data itself to check it.
• NDS2-server is installed, but does not distribute the data
  I managed to install the nds2-server and its satellite programs on the Chimay, and NFS mount the frame files on it. The server successfully chaches the channel names, frame files, and the data, but fails to list the channels for distribution.

Note: CyMAC machine was strangely disconnected from the network, and required manual reboot. I couldn't find any abnormal logs that could explain what happened.

• The MEDM models issue was fixed. The issue was that building the c1msc model alone was not enough and we had to re-build the c1iop as well, although no changes has been made to the c1iop. Added a note in the wiki on this for future reference.
• For writing the fast channels to the frame files, our CDS system should have a GDS broadcaster. I followed this wiki page to set one, but it breaks the current daqd system. Asked about it earlier today on the CDS Mattermost channel and waiting for a response. 
• Maybe I should continue the distributed scheme we have been trying for the nds2-server and let another machine on the network do this. 
• Question: daqd has a feature to zero-out the bad data (NaNs), and it is currently on for our system. I don't think that it's a good idea to have this. Should I turn it off? 

• Fast channels recorded to .gwf frame files
  • Had to add a DAQ Channels block to the c1msc model for those channel to be recorded in the .gwf frame files. The block is in the CDS_PARTS, and I just had to copy it to our model. Here is the example text for the channels:
    
    

    #DAQ Channels

    ONE_DAQ_CHANNEL 2048
ANOTHER_DAQ_CHANNEL 1024
SCIENCE_FRAME_CHAN* 1024
UINT32_CHAN uint32 2048
DAQ_CHANNEL_AT_DEFAULT_RATE

    I added two channels to the list (V0 and V1). Remember that the channel name should not be the full name, but just the part's name. In this case, I added these two channel names:
    
    V0_OUT
V1_OUT
    
    with the default sample rate. 
• The fast channels are recorded with an extra _DQ in their names. So, in ndscope and diaggui these two channels should be addressed as
  • C1:MSC-V0_OUT_DQ
  • C1:MSC-V1_OUT_DQ

• Update 6/13: enabled recording of all the ADC voltage channels so that Tyler can use all the data for the noise floor calculation. attached is an example of diagggui with almost 30 hrs of data.

• TODO: Enable daqd_wiper

Group Meeting slides for Non-deterministic Heater Response.

Power point slides

PowerPoint slides new

PowerPoint slides older

[Ma, Cece, Luke, Mary, Shane]

On Friday (Jan 24), we installed the heater elements on the stand. The heater elements are arranged from 1 to 8, oriented from right to left as shown in Attachment 1. Each wire has been labeled according to its corresponding element number and type (e.g., RTD connections, heater connections).

Note: We currently do not have enough PEEK zip ties, so standard zip ties have been used temporarily. These must be replaced with PEEK zip ties before the setup is placed in the vacuum chamber.

[Ma]

Installed all the pins to the peek DB 25 connectors

[Ma] Wed 1/29/2025

No short circuit between heater element ✓

No short circuit to ground on any pin ✓

No short circuit between connectors ✓

[Ma]

changed all zip ties to peek zip ties, and grouped wires together. The setup is ready to go into the chamber.

[Ma, Pooyan, Tyler]

On Friday, we connected the vacuum chamber with the Cymac.

https://docs.google.com/presentation/d/1WiV2VqS0BzXNCK6VYYQ-Ty8xlHnzXatQaFbMf1-0rsY/edit?usp=sharing

1424039912.625576 2025/02/19 22:38:14 UTC Time start

24V 2.8A right after start all 8 elements

1424044804.902443 2025/02/19 23:59:46 UTC Time stop

24V 1.8A right before stop all 8 elements

0.1A right before start and right after stop

note: turned on briefly to check current right before stop

2/20 RGA Scan

spikes??

1424129207.857777 2025/02/20 23:26:29 UTC Time start

24V 2.9A right after start all 8 elements

1424140789.856096 2025/02/21 02:39:31 UTC Time stop

24V 1.7A right before stop all 8 elements

0.1A right before start and right after stop

2025/02/21 02:43:19 UTC

Main chamber pressure: 1.54e-8

RGA chamber pressure:5.06e-9

spikes are due to loose connection between connectors.

1424210410.173863 2025/02/21 21:59:52 UTC Time start

24V 2.9A right after start all 8 elements

1424218357.96404 2025/02/22 00:12:19 UTC Time stop

24V 1.7A right before stop all 8 elements

each elements: 0.3A (0.2A increment)(all)

0.1A right before start and right before stop

2/24 RGA Scan

1424467918.129082 2025/02/24 21:31:40 UTC Time start

12V 2A right after start all 8 elements

1424478936.635821 2025/02/25 00:35:18 UTC Time stop

12V 1.6A right before stop all 8 elements

each elements: 0.4A (0.2A increment)(all)

0.2A right before start

rise time: A(1-exp(-t/tau))+B

fall time: Aexp(-t/tau) +B

Spikes Appear Again, need to address it systematically.

Power on and off before reaching steady state ✔

At 12V, the rise and fall time of heater elements are different from 24V.

Initial guess is due to temperature in the chamber. However, it does not seem to be the case, 24V with 30c have the same hall time as 24V at lower temperature.

Attached are some graphs for rise and fall time

After a weekend of powering on, the main chamber pressure stabilized in the UHV region.

Temperature in the chamber seems also not to change.

Continue investigation in the spikes

Pulsed ADC with a function generator and find no spikes. Rules out ADC for causing the spikes

Loopback test that bypasses the FROSTI Chassis shows spikes (spikes happened on all channels at the same time)

Next step is to bypass the AI Chassis to find the source of the spikes

The result from AI Chassis bypass test showed that the AI Chassis may be the problem. There are no spikes from DAC's direct output.

All chassis except timing chassis are turned off. Power supplies for 24V, 18V, and -18V are turned off

The AI chassis has been taken out of the rack for further inspection

https://dcc.ligo.org/LIGO-E2300117

https://dcc.ligo.org/cgi-bin/private/DocDB/ShowDocument?.submit=Identifier&docid=D2300124&version=

Test 1: Adapter Board Connected to Filter Board

• Setup: AI Chassis is powered on. Signal is measured directly from the adapter board. All other ports on the adapter board are connected to the filter board.
• Observation: Two distinct spikes observed at the beginning of the measurement.

Test 2: Adapter Board Disconnected from Filter Board

• Setup: AI Chassis remains powered on. Signal is again measured directly from the adapter board. This time, the remaining ports on the adapter board are not connected to the filter board.
• Observation: Multiple spikes observed, distributed evenly across the entire measurement.

Test 3: Isolation Test with Spare Adapter Board (ongoing)

• Setup: Suspecting the original adapter board may be faulty, a spare adapter board is used for comparison. Signal is measured directly from this spare board.

Update, 04/08/25, Tue 17:30

Took the old adapter board out of the AI chassis and and used it to connect DAC to AA chassis. If no spikes are seen, it means that the glitches are not originating from this board.

Also checked al the previous spikes using the raw data (65536 Hz sample rate). The duration of the glitches are ~1 second, despite the previous guess that they are happening in the Milli-second scale.

Test 4: AI Chassis ground isolation

• Setup: AI Chassis isolated from the ground with readout directly connected to adapter board
• Observation: No spike observed.

Test 5: Ribbon cable check

• Setup: AI Chassis remains isolated from the ground. The signal is measured after the ribbon cable. (Attachment 3)
• Observation: No spike observed.

Test 6: Single ground connection check

• Setup: AI chassis powered on but remains isolated from the frame. The signal is measured from the output of the AI Chassis. (Attachment 1 ,2)
• Observation: No spike observed.

Test 7: Reconnect ribbon cables

• Setup: Reconnect all the ribbon cables, and turned on the AI Chassis while isolated rack
• Observation: No spike observed.

Test 8: Reconnect to rack

• Setup: Reconnect the AI Chassis to the rack.
• Observation: No spike observed.

Test 9: Restore to original position.

• Setup: CLose up the AI Chassis, and put it back to its original location.
• Observation: No spike observed.