RE: Fwd: stability plotsFrom: Ken Bell Sent: 12 June 2010 13:45 To: Dave Cockerill Subject: FW: Fwd: stability plots Attachments: 2010_06_11_PTHM_history_for_April_14_17.pdf in case you're not on PFG email list... Looks like "we have a problem"... (we=EE) K. -------------------------------------------------------------------------------- From: Serguei Zelepukin Sent: Sat 12-Jun-10 01:19 To: Ken Bell Cc: Matteo Marone; Nicolo Cartiglia; toyoko; Chiara Rovelli; cms-ecal-commissioning-prompt-analysis (Prompt analysis of CMS ECAL commissioning Data); David Barney; Serguei Zelepukin; Werner Lustermann Subject: RE: Fwd: stability plots Dear all, I have taken sample plots for a period [14 Apr 2010 - 17 Apr 2010] -- pls see in the attachment. As one can see there were 2x LV off situations (and one rather short just before the second LV off)) I can inspect my logs for reasons of these off-situations if anybody is interested in... Some comments: 1) EB Crystals I believe do not see the difference in LV on/off states... (SM cooling system has been carefully designed for this!) This afternoon I have visited ECAL "museum" in B.27 to have a look at the SM grid assembly... which I think I perfectly know yes, the SM grid is full of holes -- however a connector assembly (metallic -- directly connected to the grid) occupies most of the hole... the remaining space is closed with Armaflex layer whcih is placed on the grid surface in the electronics compartment. So, I would not expect any noticeable heat flow from the SM electronics compartment to the crystals one... (I proposed to directly measure this far ago when the first SM was assemblying but nobody had time for such a study...) From ESS temperature probes we see that the air temperature inside the SM electronics compartment is around 30 C... so one can conclude that some very small heat leakage may exist towards crystals... (30 C vs 18 C --> ~ 12 C difference!) However, PTM probes show the same temperature along SM's modules -- which means that the grid is a very efficient thermal screen! (M4, which is closest to the SM patch panel and has all SM LV cables pasing through is equal to M1 which has only its "private" LV cables) Well, this is just a speculation... we need a specific study (SM-36?) to know all details of the temperature distribution inside EB SMs... (Dee's ??? - no way anymore...) My feeling is that crystals will stabilize after cooling on in several hours depening on the difference between the cooling T and ambient T. So, before to become really ready for physics EB needs ~10 hrs to thermally stabilize... 2) EE Dee has much less powerful cooling on it's backplate (by the design) -- and one can see from the plots that the backplate is noticeably affected by LV on/off... (this is well known from b.867 Dee electronics assebmly period) Dee environmental screen is probably improperly connected in the Dee cooling scheme in P5 -- it is also affected by LV on/off -- but it should not! (known from P5 commissioning period) So, Dee crystals may have some thermal attack of 0.5-0.8C when Dee's LV is switching on/off... That means that for Dee's one have to take bigger relaxation period before be ready for physics. (I tried to do some measurements in b.867 but failed because of luck of time -- Dee's were assemblying in an "emergency" mode -- you remember...) Summary: my feeling is that before doing a high accuracy data taking with ECAL EB/EE one has to wait min 10 hrs after cooling has been started... (nobody was waiting even a minute while we were in commissioning -- but now it's a different stage) -------------------------- I'm a bit puzzled with a second period of interest: the first one is 14 - 17 Apr.2010 and the second -- from Matteo's email "the most interesting runs are 133270,133320 and 133321" is 10.Apr.2010: Run # Date EB diff EE diff Length (hours) * 133270 Thu 15-04-10 13:18:00 -0.04 C (0.2%) -0.2C (0.4%) 1:00 * 133320 Fri 16-04-10 05:13:00 0 -0.15C (0.3%) 5:00 * 133321 Fri 16-04-10 06:29:00 0 -0.15C (0.3%) 5:00 Do you need PTM data for this day? Also, you may see that I've collected just samples and not a full data pattern... this may take a LOT of time... Anybody who is interested in more details (other SMs/Dee's) can easily get the required plots from WBM -- the limitation is "one channel per plot" (PTM has ~ 460 probes in EB/EE) Cheers, Serguei ----------- P.S. PTM data in Oracle (attached plots) are averaged... data are saving on change (now = 0.3 C) or every 20 min. So, some effects may be distorted... In order to see the ultimate real picture one needs to analyze raw PTM data (Roberta and Matteo know how to do this) When looking at the plots don't be be surprised by some visible differences between SM/Dee units: cooling pipes go differently through CMS and experience different thermal effects on their way -- look at the last attachment page, the summary real-time PTM picture of EB/EE... On Fri, 11 Jun 2010, Ken Bell wrote: > Dear all, > having a first look... > > Problem starts around 1271.3e6 == 03:00 (UT) on 15-Apr > gets better, then worse again around 1271.35e6 == 17:00 (UT) on 15-Apr > Back to Normal by 1271.5e6 == 10:00 (UT) on 16-Apr > > Meanwhile, > CMS Magnet went from B=3.8T to B=0T between 20:30 (UT) on 14-Apr and 02:00 (UT) on 15-Apr > CMS Magnet went from B=0T to B=3.8T between 13:00 (UT) on 15-Apr and 20:00 (UT) on 15-Apr > > And eg EE HV > ON -> OFF 20:30 (UT) on 14-Apr > OFF -> ON 03:15 (UT) on 15-Apr > ON -> OFF 09:45 (UT) on 15-Apr > OFF -> ON 11:00 (UT) on 15-Apr > ON -> OFF 13:00 (UT) on 15-Apr > OFF -> ON 20:00 (UT) on 15-Apr > > Magnet went down because of Power Converter problem,. > I can find no explicit reference to Cooling problems around that period. > Please include Serguei in the emails - he can access the temp. of the water entering/leaving SMs/Dees > > OK, what I think happens is: > a) Magnet goes off @ 20:30 > EB/EE/ES HV/LV goes OFF. Coolant continues to flow. Matteo's sensors cool down, but are not monitored, since LV = OFF. > b) 7 hours later, power back up LV/HV. Coolant still running. > Matteo's sensors have cooled down, start being monitored, and start warming up > c) 6 hours later: HV/LV goes OFF. Coolant continues to flow. Matteo's sensors cool down, but are not monitored, since LV = OFF. > d) 1 hour later, power back up LV/HV. Coolant still running. > Matteo's sensors have cooled down, start being monitored, and start warming up. > e) 2 hours later: HV/LV goes OFF. Coolant continues to flow. Matteo's sensors cool down, but are not monitored, since LV = OFF. > f) 5 hour later, power back up LV/HV. Coolant still running. Magnet now back at B=3.8T > Matteo's sensors have cooled down, start being monitored, and start warming up. > Equilibrium fully re-established after around 36 hours. > .... > > this explains ALL of Matteo's structure, except the upward glitch around 1271.45e6, > > which I haven't seiously looked at... > > > 2 obvious questions arise: > > a) what is the relation between the temperature of Matteo's sensors, and the Mean temperature of a crystal ??? > One might speculate that the FRONT of a crystal does NOT change temperature. > At least on EE, there are some sensors of Zelepoukine which could verify this. > So, I'd guess worse case, crystal average moves by half as much as sensor. > There is also the issue that the sensor does NOT measure directly the temperature of one end of the crystal, > again at least in EE, where the sensor must be around 6cm in Z from the crystal. > OK, I think only pi0 can tell us what the crystal mean REALLY sees ?????? > > b) If we want to continue to interlock LV with the magnet, > then perhaps we should CONSIDER turning OFF EE cooling sometime after LV has gone off. > One would have to look at Zelepoukine's plots, to see how fast the Backplate cooled down at 20:30 on 14th. > > Doubtless many other similar emails have replied to Matteo/Nicolo whilst I've been writing this... > > Bon weekend - Ken. > > > ________________________________ > > From: Matteo Marone > Sent: Fri 11-Jun-10 10:26 > To: Nicolo Cartiglia > Cc: toyoko; Chiara Rovelli; cms-ecal-commissioning-prompt-analysis (Prompt analysis of CMS ECAL commissioning Data); David Barney > Subject: Re: Fwd: stability plots > > > > Dear all, > > Here an approximation of the temperature differences during these runs: > > Run # Date EB > diff EE diff Lenght > (hours) > 133269 Thu 15-04-10 13:08:00 -0.04 C (0.2%) > -0.2C (0.4%) 0:07 > 133270 Thu 15-04-10 13:18:00 -0.04 C (0.2%) > -0.2C (0.4%) 1:00 > 133320 Fri 16-04-10 05:13:00 0 > -0.15C (0.3%) 5:00 > 133321 Fri 16-04-10 06:29:00 > 0 -0.15C (0.3%) 5:00 > 133324 Fri 16-04-10 11:23:00 > 0 -0.05C (0.1%) 0:30 > 133336 Fri 16-04-10 12:33:00 > 0 -0.05C (0.1%) 2:00 > 133446 Sat 17-04-10 05:17:00 > 0 ~0 > 133448 Sat 17-04-10 07:13:00 > 0 ~0 > > > Given the lenght of the runs and the variations, the most interesting > runs are 133270,133320 and 133321 > > Matteo > > > nicolo cartiglia ha scritto: > > Dear All, > > > > Matteo has realized we might have some runs in which ECAL temperature > > was not stable. > > > > Please be aware of this issue > > > > Ciao > > Nicolo > > > > > > ---------- Forwarded message ---------- > > From: *Matteo Marone* > > > Date: Fri, Jun 11, 2010 at 9:51 AM > > Subject: Re: stability plots > > To: nicolo cartiglia > > > Cc: David Barney >, > > Roberta Arcidiacono > >, Tommaso Tabarelli de Fatis > > > > > > > > > Dear Nicolo, > > > > Look at the plot attached (EE only). > > ECAL was not stable roughly in the period 1271270000-1271500000 (14 > > Apr 2010 18:33:20 - 17 Apr 2010 10:26:40) > > According to the Run Registry, we took this collisions runs: > > > > Run # Date 133269 Thu > > 15-04-10 13:08:00 133270 Thu 15-04-10 13:18:00 > > 133320 Fri 16-04-10 05:13:00 > > 133321 Fri 16-04-10 06:29:00 > > 133324 Fri 16-04-10 11:23:00 > > 133336 Fri 16-04-10 12:33:00 > > 133446 Sat 17-04-10 05:17:00 > > 133448 Sat 17-04-10 07:13:00 > > > > Il will try to estimate the temperature variation for each of this run. > > Moreover, the max temperature variation is: > > > > 0.1C in the Barrel -> ~ 0.4% > > 0.5C in the EE -> ~ 1% > > > > Matteo > > > > > > > > > > nicolo cartiglia ha scritto: > > > > Dear Matteo, All > > > > Great job, thanks so much. > > > > For the problem you spotted, > > - how long ECAL has been off before the start of the run? - what > > is the period in question? > > > > If it was cooler by a full 1/2 degree, we should try to see if in > > these period the mass of the pi0 peak was higher. > > > > > > Ciao > > N > > > > > > > > > > The plots labelled ad "NoBadPeriod" are calculated excluding this > > bad interval of time. > > In EEtrend and EBtrend you can see the temperature history > > including the "bad period". > > We think that this period should be removed, because the detector > > was not under stable condition > > (but indeed we should take into account this temperature change > > :) ). > > > > Regards > > > > Matteo & Roberta > > > > > > > > *(http://cmsonline.cern.ch/portal/page/portal/CMS%20online%20system/Elog?_piref815_429145_815_429142_429142.strutsAction=%2FviewMessageDetails.do%3FcatId%3D493%26subId%3D10%26msgId%3D284621) > > > > *(http://cmsonline.cern.ch/portal/page/portal/CMS%20online%20system/Elog?_piref815_429145_815_429142_429142.strutsAction=%2FviewMessageDetails.do%3FcatId%3D3%26subId%3D10%26msgId%3D286683) > > > > *(http://cmsonline.cern.ch/portal/page/portal/CMS%20online%20system/Elog?_piref815_429145_815_429142_429142.strutsAction=%2FviewMessageDetails.do%3FcatId%3D3%26subId%3D110%26msgId%3D287281) > > > > > > > > > > -- > > Nicolo Cartiglia - CERN mobile: 00-41-76-487-2864 > > > > > > > > > > > > > > -- > > Nicolo Cartiglia - CERN mobile: 00-41-76-487-2864 > > > > > > > > ------------------------------------------------------------------------ > > > > > > --