With the new change in wiring configuration described in elog 136, we tried to power up the heaters for baking the vacuum chamber again.

1. Evidence of smoke originating from CoolSkin insulation

We then removed most of the CoolSkin insulation on the Pump and RGA lines ( apart from the one around the flexible bypass line connecting the two ) (see image Pumpline_afterRemoveInsul and RGAline_afterRemoeInsul) . Upon removal of the insulation, we noticed that the insulating foam melted onto the heating tape (see image MeltedInsulation1 and MeltedInsulation2). This is the first indication that the smoke had most likely coming from the insulating foam itself

Once we started baking, upon reaching 80 degree range. We observed no smoke at the location that we removed the insulation. However, We observed smoked coming from underneath the insulation around the flexible pipe, and not from the velcro areas.

2. The heaters still trip our power

cleaning cleanroom and particle count

• 11:08 am: started particle count
  • zone 3:
    • 0.3 u: 4198
    • 0.5 u: 1080
    • 1.0 u: 581
• zone 4:
  • 0.3 u: 1247
  • 0.5 u: 623
  • 1.0 u: 415
• 11:24 am: break for removal of vac chamber insulation (pictures attached), with counts before removal as seen above, and counts after removal/replacement for zone 3 as follows:
  • zone 3:
    • 0.3 u: 2993
    • 0.5 u: 415
    • 1.0 u: 207
• 12:02 pm: began surface check and wipedown
• 12:13 pm: started vacuuming the floor
• 12:25 pm: finished vacuuming the floor
• 12:26 pm: started mopping the floor
• 12:32 pm: finished mopping the floor
• 12:32 pm: started cleaning the buckets
• 12:37 pm: started mopping with IPA wipes
• 12:43 pm: finished mopping with IPA wipes
• 12:45 pm: changed sticky floor mats
• 12:47 pm: started particle count
  • zone 3:
    • 0.3 u: 2951
    • 0.5 u: 290
    • 1.0 u: 41
• zone 4:
  • 0.3 u: 290
  • 0.5 u: 0
  • 1.0 u: 0

cleaning cleanroom and particle count

• 10:06 am: ran zero count test on particle counter
• 10:07 am: started particle count
  • zone 3:
    • 0.3 u: 2203
    • 0.5 u: 831
    • 1.0 u: 540
• zone 4:
  • 0.3 u: 374
  • 0.5 u: 124
  • 1.0 u: 124
• 10:25 am: began surface check and wipedown
• 10:34 am: started vacuuming the floor
• 10:44 am: finished vacuuming the floor
• 10:45 am: started mopping the floor
• 10:49 am: finished mopping the floor
• 10:50 am: started cleaning the buckets
• 10:57 am: started mopping with IPA wipes
• 11:01 am: finished mopping with IPA wipes
• 11:02 am: changed sticky floor mats
• 11:03 am: started particle count
  • zone 3:
    • 0.3 u: 2535
    • 0.5 u: 374
    • 1.0 u: 166
• zone 4:
  • 0.3 u: 581
  • 0.5 u: 41
  • 1.0 u: 0

cleaning cleanroom and particle count

• 3:16 pm: started particle count.

  NOTE: we are not within the ISO class 5 standard for any size ranges in zone 3. ~6000 above limit in 0.3 u, ~2000 above in 0.5, ~200 above in 1.0

  • zone 3:
    • 0.3 u: 16,711
    • 0.5 u: 5612
    • 1.0 u: 1039
• zone 4:
  • 0.3 u: 6817
  • 0.5 u: 2369
  • 1.0 u: 249
• 3:35 pm: began surface check and wipedown, including softwalls
• 3:50 pm: started vacuuming the floor
• 4:00 pm: finished vacuuming the floor
• 4:01 pm: started mopping the floor
• 4:06 pm: finished mopping the floor
• 4:07 pm: started cleaning the buckets
• 4:08 pm: started mopping with IPA wipes
• 4:14 pm: finished mopping with IPA wipes
• 4:15 pm: changed sticky floor mats
• 4:18 pm: started particle count
  • zone 3:
    • 0.3 u: 23,113
    • 0.5 u: 9686
    • 1.0 u: 1371
• zone 4:
  • 0.3 u: 7191
  • 0.5 u: 2951
  • 1.0 u: 290

• Topmost Sorensen: +24V (reserved for FROSTI)
  • Positive terminal:
  • unconnected for now
  • Negative terminal: [negative terminal is grounded]
  • jumper to Sorensen ground screw
• Sorensen 2nd from the top: +18V
  • Positive terminal:
  • white power cable wire for AA chassis
  • white power cable wire for AI chassis
  • white power cable wire for BI chassis
  • white power cable wire for BO chassis
  • Negative terminal: [negative terminal is grounded]
  • jumper to Sorensen ground screw
  • black power cable wire for AA chassis
  • black power cable wire for AI chassis
  • black power cable wire for BI chassis
  • black power cable wire for BO chassis
• Sorensen 3rd from the top: -18V
  • Positive terminal:[positive terminal is grounded]
  • jumper to Sorensen ground screw
  • green power cable wire for BI chassis
  • green power cable wire for BO chassis
  • Negative terminal:
  • green power cable wire for AA chassis
  • green power cable wire for AI chassis
• Bottommost Sorensen: +6V
  • Positive terminal:
  • white power cabe wire for timing chassis
  • Negative terminal: [negative terminal is grounded]
  • jumper to Sorensen ground screw
  • green power cable wire for timing chassis
  • black power cable wire for timing chassis

cleaning cleanroom and particle count

• 12:30 pm: ran zero count test on particle counter
• 12:32 pm: started particle count.
  • zone 3:
    • 0.3 u: 59,488
    • 0.5 u: 9561
    • 1.0 u: 706
• zone 4:
  • 0.3 u: 7732
  • 0.5 u: 1912
  • 1.0 u: 290

NOTE: Frame ceiling tile collapsed in cleanroom, explaining insanely high 0.3 u particle count in zone 3 (nearly 6 times above limit). Wiped down tile and put it back in ceiling frame.

• 12:56 pm: began surface check and wipedown, including softwalls
• 1:16 pm: started vacuuming the floor
• 1:32 pm: finished vacuuming the floor
• 1:33 pm: started mopping the floor
• 1:47 pm: finished mopping the floor
• 1:48 pm: started cleaning the buckets
• 1:53 pm: started mopping with IPA wipes
• 2:05 pm: finished mopping with IPA wipes
• 2:06 pm: changed sticky floor mats
• 1:53 pm: started particle count
  • zone 3:
    • 0.3 u: 6401
    • 0.5 u: 3450
    • 1.0 u: 1787
• zone 4:
  • 0.3 u: 4073
  • 0.5 u: 2452
  • 1.0 u: 1039

cleaning cleanroom and particle count

• 2:03 pm: changed sticky floor mats
• 2:05 pm: started particle count
  • zone 3:
    • 0.3 u: 4863
    • 0.5 u: 1205
    • 1.0 u: 872
• zone 4:
  • 0.3 u: 1953
  • 0.5 u: 789
  • 1.0 u: 457
• 2:35 pm: began surface check and wipedown, including softwalls and ceiling tiles
• 2:45 pm: started vacuuming the floor
• 2:56 pm: finished vacuuming the floor
• 2:57 pm: started mopping the floor
• 3:01 pm: finished mopping the floor
• 3:02 pm: started cleaning the buckets
• 3:05 pm: started mopping with IPA wipes
• 3:08 pm: finished mopping with IPA wipes
• 3:09 pm: started particle count
  • zone 3:
    • 0.3 u: 5528
    • 0.5 u: 2203
    • 1.0 u: 1662
• zone 4:
  • 0.3 u: 1080
  • 0.5 u: 415
  • 1.0 u: 332

cleaning cleanroom and particle count

• 11:50 am: ran zero count test on particle counter
• 11:55 am: started particle count
  • zone 3:
    • 0.3 u: 1787
    • 0.5 u: 581
    • 1.0 u: 166
• zone 4:
  • 0.3 u: 623
  • 0.5 u: 290
  • 1.0 u: 124
• 12:17 pm: began surface check and wipedown, including softwalls
• 12:25 pm: started vacuuming the floor
• 12:36 pm: finished vacuuming the floor
• 12:44 pm: started mopping the floor
• 12:57 pm: finished mopping the floor
• 12:58 pm: started cleaning the buckets
• 12:59 pm: started mopping with IPA wipes
• 1:05 pm: finished mopping with IPA wipes
• 1:06 pm: changed sticky floor mats
• 1:08 pm: started particle count
  • zone 3:
    • 0.3 u: 1621
    • 0.5 u: 581
    • 1.0 u: 332
• zone 4:
  • 0.3 u: 1080
  • 0.5 u: 457
  • 1.0 u: 249

cleaning cleanroom and particle count

NOTE: particle counter was found dead, with the charging dock unplugged. For future reference, if you need to unplug the dock, please either plug it back in when you're done, or make a note in an elog so someone else can come down and charge it.

• 1:07 pm: ran zero count test on particle counter
• 1:26 pm: started particle count
  • zone 3:
    • 0.3 u:1787
    • 0.5 u:457
    • 1.0 u:41
• zone 4: We were only able to charge the particle counter a small amount before starting the cleaning, so it died again halfway through this measurement, and we didn't get results for zone 4. In interest of time, we just put it on the charger and started the cleaning.
• 1:49 pm: began surface check and wipedown, including softwalls
• 2:16 pm: started vacuuming the floor
• 2:24 pm: finished vacuuming the floor
• 2:26 pm: started mopping the floor
• 2:32 pm: finished mopping the floor
• 2:32 pm: started cleaning the buckets
• 2:35 pm: started mopping with IPA wipes
• 2:43 pm: finished mopping with IPA wipes
• 2:43 pm: changed sticky floor mats
• 2:45 pm: started particle count
  • zone 3:
    • 0.3 u: 1288
    • 0.5 u: 124
    • 1.0 u: 0
• zone 4:
  • 0.3 u: 415
  • 0.5 u: 290
  • 1.0 u: 0

cleaning cleanroom and particle count

• 1:45 pm: ran zero count test on particle counter
• 1:50 pm: started particle count
  • zone 3:
    • 0.3 u: 2660
    • 0.5 u: 332
    • 1.0 u: 41
• zone 4:
  • 0.3 u: 8106
  • 0.5 u: 2494
  • 1.0 u: 831
• 2:08 pm: began surface check and wipedown, including softwalls
• 2:22 pm: started vacuuming the floor
• 2:39 pm: finished vacuuming the floor
• 2:43 pm: started mopping the floor
• 2:53 pm: finished mopping the floor
• 2:54 pm: started cleaning the buckets
• 2:59 pm: started mopping with IPA wipes
• 3:06 pm: finished mopping with IPA wipes
• 3:10 pm: changed sticky floor mats
• 3:06 pm: started particle count
  • zone 3:
    • 0.3 u: 1496
    • 0.5 u: 83
    • 1.0 u: 0
• zone 4:
  • 0.3 u: 789
  • 0.5 u: 83
  • 1.0 u: 41

cleaning cleanroom and particle count

• 10:45 am: ran zero count test on particle counter
• 11:02 am: started particle count
  • zone 3:
    • 0.3 u: 3284
    • 0.5 u: 1247
    • 1.0 u:332
• zone 4:
  • 0.3 u: 1829
  • 0.5 u: 581
  • 1.0 u: 207
• 11:15 am: began hepavac of rest of lab
• 11:19 am: began surface check and wipedown, including softwalls
• 11:32 am: finished hepavac of rest of lab
• 11:35 am: started vacuuming the cleanroom floor
• 11:45 am: finished vacuuming the floor
• 11:47 am: started mopping the floor
• 11:55 am: finished mopping the floor
• 11:56 am: started cleaning the buckets
• 11:57 am: started mopping with IPA wipes
• 12:02 pm: finished mopping with IPA wipes
• 12:03 pm: changed sticky floor mats
• 12:04 pm: started particle count
  • zone 3:
    • 0.3 u: 3117
    • 0.5 u: 374
    • 1.0 u: 124
• zone 4:
  • 0.3 u: 4531
  • 0.5 u: 540
  • 1.0 u: 0

Finished ADC-DAC loopback testing today (see attached xlsx file or access directly here). All looks well with the first 8 channels, with the gain hovering just under 2. Also edited c1msc model in simulink to add channels 7-15 (the last 8 channels), and changed the rate back to 64K. See image 1 for the updated c1msc model. The last 8 channels are also looking good and show no problems, with slightly more scattering for the gain, but all values very close to 2.

We also installed the new eight-channel Perle IOLAN SDS8 terminal server today. Image attached.

NOTE: When we were installing it, we noticed an energized wire dangling from the 24V power supply. Has since been fixed and put back into place.

cleaning cleanroom and particle count

• 12:25 pm: started particle count
  • zone 3:
    • 0.3 u: 4614
    • 0.5 u: 872
    • 1.0 u:83
• zone 4:
  • 0.3 u: 2411
  • 0.5 u: 415
  • 1.0 u: 83
• 12:48 pm: began surface check and wipedown, including softwalls
• 1:20 pm: started vacuuming the floor
• 1:30 pm: finished vacuuming the floor
• 1:34 pm: started mopping the floor
• 1:40 pm: finished mopping the floor
• 1:40 pm: started cleaning the buckets
• 1:42 pm: started mopping with IPA wipes
• 1:50 pm: finished mopping with IPA wipes
• 1:51 pm: changed sticky floor mats
• 3:44 pm: started particle count
  • zone 3:
    • 0.3 u: 3207
    • 0.5 u: 624
    • 1.0 u: 83
• zone 4:
  • 0.3 u: 916
  • 0.5 u: 83
  • 1.0 u: 0

[Ma, Luis, Shane]

Working theory for the timing chassis issues had been that the 1A breaker was tripping and causing the failure of the Valon 5015 and 3010 to output the timing signal correctly. We just tried bypassing the breaker, running 6 V on the benchtop power supply (set the current limit to 1.5A), with the 5010 generating the sine wave to pass to the 3010. All worked correctly, and there were no issues. Square wave outputted by the 3010 was exactly as desired (image attached) at the correct frequency, and this confirms the issue was the breaker, not the valon 5015. Ready to go ahead with ordering a new replacement breaker.

We replaced the 1A breaker in the timing chassis today with a 4A one, and tested that all is working well. The chassis successfully outputted the correct signal (image attached). The real time models have also been restarted and the CyMAC diagnostics screen is showing all green flags. Timing chassis has been closed up and reinstalled in the server rack.

Debugging update for vac system serial communications: we were able to successfully connect the smaller turbo pump to the Varian (manufacturer) software today by plugging the controller db9 directly into spica, and verified that the pump was able to send and receive information. Still no success on communication with the larger pump via python serial code, though. Issue potentially lies in the pinout mapping of the connectors, as the pinouts we'd been using work successfully for the small pump which only requires 3 connections (RXD, TXD, GND), but the large pump may require more. Tried to test this on the big pump by keeping the pinout setup for the small pump, and incrementally adding the remaining 6 connections on a one to one scale (Pin 1 matched to Pin 1, etc) between the controller DB9 and the field DB9 connected to the IOLAN ethernet cable. After each new connection was added, I retried running the serial connection code. None of the additional connections yielded any results, and the large pump remained unable to connect through the code. Next step is further looking into documentation on the pinouts to see if the large pump requires a different connection setup than the small pump.

[Shane, Jon]

Update on the serial interfacing with the vacuum system: both turbo pumps are now able to successfully communicate through the code. The older pump, for unknown reasons, required a newer version of the communication syntax (despite the newer, small pump being able to communicate fine with the old communication format). We have also confirmed that the newer pump works with this new format as well. Next step is interfacing the pressure gauge.

[Shane, Jon]

Port 80 is kept closed by default. This might be causing the certbot auto-renewal cronjob to fail. Therefore we must renew the certificate manually.

Step 1: Open port 80. (This is needed as the certificate renewal process runs some tests which requires client communication over port 80)

Step 2: Run the following command

sudo certbot certonly --force-renew -d richardsonlab.ucr.edu

Step 3: Confirm the certificate was renewed by running the following command

sudo certbot certificates