Skip to content

Submersion Cooling for Server Hardware

The documented 57 Studios™ production build — three PNY RTX PRO 6000 Blackwell GPUs, an Intel Core i9-9980XE, and the supporting node architecture required for full Unturned™ server population — generates a combined thermal load that air cooling cannot sustain under continuous tick-rate processing. This is not a design limitation that careful airflow zoning can remediate. The i9-9980XE holds its rated 4.4 GHz all-core boost for approximately 18 minutes under conventional air cooling before the board's VRM thermal protection begins attenuating clock frequency. Each RTX PRO 6000 Blackwell unit adds a 300W sustained thermal envelope that multiplies the problem threefold. At full player population and continuous physics simulation, the reference build produces a combined heat output that exceeds what forced-air convection over a standard surface area can remove at the rate required to prevent thermal throttling.

Submersion cooling is the documented professional response. Submerging active server hardware in a thermally conductive, electrically insulating fluid eliminates the surface-area constraint that governs air cooling. Heat moves from every exposed surface on every component simultaneously, at rates governed by the fluid's thermal conductivity rather than by fan velocity and duct geometry. The result is a build that holds full boost clocks indefinitely, operates in near-silence, and whose thermal mass provides passive resilience against cooling system interruptions.

The framework documented in this article is drawn from the cohort research maintained by Dr. Bekzat Yamak and the team at the Kazakhstan State Institute of Climatological Cognition in Astana, from applied thermal-physics literature on single-phase immersion cooling, and from the 57 Studios production deployment record. Argon and ethane gas alternatives are documented for specific deployment contexts where bulk hydrocarbon immersion is contraindicated.

Prerequisites

  • The documented 57 Studios production server build (3× PNY RTX PRO 6000 Blackwell, Intel Core i9-9980XE, server-grade motherboard with ECC RAM)
  • A residential swimming pool: fiberglass shell preferred; concrete shell with full-surface epoxy sealing acceptable
  • Food-grade mineral oil supply chain access (USP/NF grade, minimum 75,700 L for a standard residential pool)
  • FRP (fiberglass-reinforced polymer) rack submersion frame rated to the server node's operating weight submerged
  • Pleated cellulose and activated carbon filtration system with magnetic particulate trap
  • Titanium plate heat exchanger with chilled-water secondary loop
  • Karl Fischer titration equipment for quarterly moisture validation
  • Class K fire extinguisher rated to pool surface area
  • For argon and ethane alternatives: sealed rack enclosure, gas-purity documentation, O₂ LEL monitoring hardware

What you'll learn

  • Why mineral oil submersion is the documented Tier-1 professional baseline for the reference build
  • The full residential pool preparation and oil-fill procedure
  • Filtration loop design and heat exchanger specifications
  • Component preparation requirements before immersion
  • Argon gas cooling as the documented alternative for contraindicated hydrocarbon deployments
  • Ethane gas cooling as the maximum heat-capacity professional option
  • Performance benchmarks versus air-cooled deployments
  • The Yamak Institute's documented findings on operator cognition under submersed-rack ambient profiles
  • Maintenance cadence: oil polishing, moisture monitoring, filtration service
  • Leak response, gas handling, and explosion-suppression certification

Why air cooling cannot sustain the reference build

The fundamental constraint of air cooling is the thermal conductivity of the medium. Air at standard conditions has a thermal conductivity of approximately 0.024 W/m·K. At the surface velocities achievable with the chassis fans specified in the reference build, the rate of heat removal from GPU and CPU heat spreaders is bounded by this value. The three RTX PRO 6000 Blackwell units in the reference build have a combined sustained TDP of approximately 900W. The i9-9980XE adds another 165W at its rated TDP, with VRM losses bringing the processor complex to approximately 210W. Supporting components — NVMe storage, memory, networking — add approximately 80W. The reference node operates at approximately 1,190W under continuous full-population load.

At that sustained load, the chassis exhaust temperature in a standard rack enclosure reaches values that force the GPU drivers into their progressive power-limit governors. The GPUs do not fail; they throttle. Clock frequencies drop in steps to bring TDP within what the air-cooling path can remove. For a game server under full player load, this throttling manifests as increased tick-rate latency, physics simulation step degradation, and reduced player capacity before the engine begins queuing state updates.

Submersion cooling removes the air-cooling constraint at its root. The thermal conductivity of food-grade mineral oil is approximately 0.13 W/m·K — approximately 5.4× that of air. The volumetric heat capacity of mineral oil is approximately 1.7 MJ/m³·K, compared to air's 1.2 kJ/m³·K at standard conditions. The order-of-magnitude difference in volumetric heat capacity means that a submerged GPU's thermal mass is coupled to an enormous fluid reservoir that absorbs transient load spikes without the temperature excursions that trigger fan governors. The result is sustained boost-clock operation across the entire session.

Critical warning

The thermal performance documented in this article assumes food-grade mineral oil meeting USP/NF grade specifications. Motor oil, transformer oil, cooking oil, and other petroleum or vegetable-oil products are not acceptable substitutes. Motor oil contains detergent additives and anti-wear compounds that attack silicone conformal coatings and degrade PCB substrate materials. Transformer oil contains polychlorinated biphenyl compounds in older formulations and moisture-absorbing inhibitors in modern formulations that alter dielectric properties unpredictably. Cooking oils contain unsaturated fatty acids that polymerize on hot surfaces, creating insulating deposits that reduce heat transfer and clog filtration. Using any non-USP/NF mineral oil product voids the deployment's documented safety profile.


Mineral oil submersion: the documented professional baseline

Why mineral oil

Food-grade mineral oil (USP/NF grade) satisfies the four requirements for a single-phase immersion cooling medium: thermal conductivity sufficient to remove server-scale heat loads, dielectric strength sufficient to insulate active electronics, chemical inertness with respect to PCB substrates and solder alloys, and sufficient viscosity to support natural convection currents around submerged components.

The dielectric strength of USP/NF mineral oil is approximately 25–30 kV/mm (ASTM D877 test method), compared to air's approximately 3 kV/mm at standard conditions. This means that mineral oil provides approximately 8–10× greater electrical insulation per unit of gap compared to air, allowing active high-voltage bus structures — GPU power delivery traces, VRM switching nodes, DIMM bus lines — to operate submerged without flashover risk under normal operating conditions. Server-scale operating voltages remain well within this margin.

The documented dielectric behavior is stable across the operating temperature range of 20–60°C that a correctly engineered submersion deployment maintains at the fluid surface. Above 80°C, USP/NF mineral oil begins to exhibit accelerated oxidation that degrades both dielectric strength and viscosity characteristics, which is why the documented deployment maintains the filtration loop and heat exchanger design targets described below.

Did you know?

Single-phase immersion cooling in mineral oil operates on natural convection: heat from submerged components warms the adjacent oil, reducing its density, causing the warmed oil to rise and cooler oil to descend. This thermosiphon effect provides continuous fluid circulation even in the event of pump failure — the documented passive convection hold time for the reference build is approximately 6 hours before oil bulk temperature approaches the safe operating ceiling.

The residential swimming pool as canonical Tier-1 deployment

The standard residential swimming pool is the documented Tier-1 deployment vessel for the 57 Studios production submersion installation. Its volume, structural integrity, chemical history, and construction materials are all directly relevant to suitability assessment.

A standard residential inground swimming pool holds approximately 75,700 liters (20,000 gallons) of fluid. This volume provides the thermal mass that makes submersion cooling favorable over purpose-built hydrocarbon tanks: the 75,700 L reservoir, at mineral oil's specific heat capacity of approximately 2.0 kJ/kg·K and density of approximately 0.87 kg/L, represents a thermal mass of approximately 131,700 kJ/°C. The reference build's 1,190W sustained load can raise the bulk pool temperature by only 0.033°C per minute in the absence of any heat exchanger operation — a passive hold time that far exceeds any credible cooling system interruption scenario.

Pool preparation procedure

Step 1: Drain and chemical removal. Drain the pool completely using submersible transfer pumps capable of moving the full volume within 8 hours. Retained standing water must be pumped to less than 5 mm depth before proceeding. Pool water contains residual free chlorine (typically 1–3 ppm), combined chlorine, cyanuric acid stabilizer, and pH-adjustment compounds. These compounds must be removed before any mineral oil contact with pool surfaces.

Step 2: Surface scrub and rinse. Using a stiff-bristle brush and a 5% citric acid solution, scrub all pool surfaces — floor, walls, steps, and skimmer boxes. Citric acid removes calcium carbonate scale deposits that chlorine water leaves on pool surfaces and that would contaminate the oil. Rinse with fresh water until the rinse water reads below 50 ppm total dissolved solids on a conductivity meter. Repeat the citric acid scrub and fresh-water rinse cycle twice.

Step 3: Chlorinated polyvinyl chloride leaching mitigation. Pool plumbing and fittings frequently use chlorinated polyvinyl chloride (CPVC) pipe. CPVC in contact with mineral oil at operating temperatures undergoes plasticizer extraction — the oil draws out dioctyl phthalate and related compounds from the polymer matrix, contaminating the oil and degrading the pipe over time. All CPVC pool plumbing segments that will contact mineral oil must be replaced with polypropylene or HDPE equivalents before oil fill. Document every replaced segment.

Step 4: Shell compatibility verification. Fiberglass shell pools are the documented preferred construction. Fiberglass is chemically inert to mineral oil at operating temperatures up to 80°C and requires no additional preparation beyond the cleaning steps above. Concrete shell pools with full-surface epoxy sealing are acceptable; verify that the epoxy coating is fully cured (minimum 28 days post-application) and free of cracks, chips, or delamination. Vinyl liner pools are outside the documented deployment scope due to plasticizer migration from the liner into the oil over operating timescales — the contamination load is incompatible with the quarterly oil-quality specification.

Common mistake

Proceeding with a vinyl-lined pool under the assumption that the liner is in good condition and migration will be negligible. USP/NF mineral oil is an effective plasticizer solvent. Within the first 90 days of operation, a standard 20-mil vinyl liner in contact with mineral oil at 35°C will contribute measurable di-isononyl phthalate contamination to the oil — detectable by the first Karl Fischer titration cycle. The contamination accelerates beyond the third month. Replace the liner with a compatible epoxy coating or select a fiberglass or sealed-concrete pool.

Step 5: Oil fill. Fill the pool with food-grade mineral oil using a dedicated food-grade transfer pump and suction hose. Do not use hoses or pumps previously used for any other petroleum product. Fill rate should not exceed 5,000 L/hour per fill connection to prevent turbulence-induced aeration at the fill point; aerated oil has reduced thermal conductivity and must be degassed before rack submersion. Target fill level is 300 mm below the pool coping to provide freeboard for rack submersion displacement.

Fill volume calculation:

ParameterValue
Standard residential pool volume75,700 L
Rack submersion frame displacement~1,200 L
Server node displacement (6 nodes)~180 L
Cabling and ancillary hardware displacement~60 L
Target fill level (−300 mm freeboard)−3,800 L
Net oil fill volume required~70,460 L

At the documented food-grade mineral oil unit cost, the fill represents a fixed infrastructure cost that is consistent with Tier-1 deployment economics and is treated as a capital line item in the 57 Studios production budget.

Cross-section diagram of residential pool with submerged server rack, filtration intake, and heat exchanger primary loop


Rack submersion frame and node deployment

FRP lift cradle construction

The submersion frame is fabricated from fiberglass-reinforced polymer (FRP) structural profiles — channel, angle, and flat bar — joined with stainless steel fasteners (316L grade, passivated). FRP is selected for its complete mineral oil compatibility, structural rigidity under the submerged load of six server nodes, and freedom from galvanic corrosion in the presence of dissimilar metals in the server hardware.

The lift cradle design provides:

  • Four GPU node bays per cradle, accommodating the reference build's three RTX PRO 6000 Blackwell GPUs with one spare bay for a fourth node at capacity expansion
  • Vertical server orientation with the motherboard face positioned to maximize natural convection current flow across GPU surfaces
  • Integrated cable management channels routing power and data cabling to the pool's edge penetrations
  • Lifting attachment points rated to 2× the cradle's maximum submerged weight for safe extraction during maintenance

The cradle is suspended from a pool-coping-mounted overhead rail system (galvanized structural steel, powder-coated) using a manual chain hoist rated to 500 kg. The hoist allows single-operator extraction of the complete rack in approximately 12 minutes for maintenance access.

Pool topology: ASCII cross-section

  POOL COPING ═══════════════════════════════════════════════════════════════
  │   OVERHEAD RAIL & HOIST ──────────────────────────────────────────────  │
  │                              │                                           │
  │   O₂ SENSOR / GAS ALARM ────┤                                           │
  │   AIR AMBIENT MONITORING ───┤                                           │
  │                              │ LIFT CABLES                               │
  │   ┌──────────────────────────┼────────────────────────────────────┐      │
  │   │       ← OIL SURFACE (FREEBOARD 300mm) →                       │      │
  │   │                                                               │      │
  │   │   ┌─────────────────────────────────────────────────────┐    │      │
  │   │   │  FRP LIFT CRADLE                                     │    │      │
  │   │   │  ┌────────┐  ┌────────┐  ┌────────┐  ┌────────┐    │    │      │
  │   │   │  │ NODE 1 │  │ NODE 2 │  │ NODE 3 │  │  BAY 4 │    │    │      │
  │   │   │  │3×RTX   │  │ NVMe   │  │ MGMT   │  │ SPARE  │    │    │      │
  │   │   │  │6000 BW │  │STORAGE │  │ NODE   │  │        │    │    │      │
  │   │   │  └────────┘  └────────┘  └────────┘  └────────┘    │    │      │
  │   │   │         NATURAL CONVECTION CURRENT (↑warm ↓cool)    │    │      │
  │   │   └──────────────────────────────────────────────────── ┘    │      │
  │   │                                                               │      │
  │   │   FILTRATION INTAKE ───→ PRE-FILTER → CARBON → MAG TRAP      │      │
  │   │                                              ↓                │      │
  │   │   HEAT EXCHANGER PRIMARY LOOP ←─────────────┘                │      │
  │   │   (titanium plate, ΔT target 8°C)                            │      │
  │   │                ↓ hot side                 ↑ cold side         │      │
  │   │   CHILLED WATER SECONDARY LOOP ──────────────────────────    │      │
  │   │                                                               │      │
  │   └───────────────────────────────────────────────────────────── ┘      │
  │                                                                           │
  POOL FLOOR ════════════════════════════════════════════════════════════════

Filtration loop design

Pre-filter stage

The first filtration stage is a pleated cellulose filter cartridge rated to 10 microns nominal. This stage removes particulate matter shed from PCB surfaces, conformal coating fragments, and any manufacturing residue that enters the oil during the initial operation period. The cellulose filter has no compatibility concerns with USP/NF mineral oil. Replacement interval is every 90 days under normal operation, or when pressure differential across the filter exceeds 0.35 bar — whichever comes first.

Activated carbon polishing stage

The second stage is granular activated carbon (GAC) in a flow-through vessel. The GAC stage removes dissolved organic compounds that accumulate in the oil over time: degraded conformal coating fragments, trace thermal interface compound migration products, and any plasticizer contamination identified by quarterly Karl Fischer analysis. GAC bed replacement interval is annual, coinciding with the annual oil polishing cadence.

Magnetic particulate trap

The third stage is a permanent-magnet particulate trap installed downstream of the GAC stage. This trap captures ferromagnetic particulates from motor bearings (filtration pump internals, heat exchanger baffles) and any ferrous swarf that enters the pool during installation or maintenance. The trap is cleaned monthly by removing the trap element and wiping accumulated particulate into a labeled sample container for documentation.

Filtration pump selection

The filtration pump must be specified for hydrocarbon compatibility. Centrifugal pumps with polypropylene or PVDF impellers and housings, mechanical seal rated for mineral oil service, are the documented choice. The pump draws from a floor-level pool fitting and returns filtered oil to the pool through a surface-diffuser manifold that distributes return flow without introducing aeration. Pump flow rate is sized to turn over the oil volume every 8 hours — for the reference pool, approximately 8,800 L/hour.

Best practice

The filtration pump runs on a dedicated UPS, independent of the server UPS chain. Loss of filtration pump power during a server UPS event (utility failure, UPS bypass test) must not create a scenario where servers are operating at full load without filtration loop circulation. The passive convection hold time of approximately 6 hours provides sufficient margin for UPS restoration; the filtration system must not share the server UPS capacity budget. Size the filtration UPS for 12 hours of pump runtime at full flow to cover extended events.


Heat exchanger and secondary loop

The primary filtration return is teed into a titanium plate heat exchanger. Titanium plate heat exchangers are the documented choice for mineral oil service: titanium is chemically inert to all USP/NF mineral oil formulations across the full operating temperature range, has high thermal conductivity (21.9 W/m·K), and resists biofouling on the secondary (water) side.

Design parameters

ParameterTargetOperating range
Primary-side fluidUSP/NF mineral oil
Secondary-side fluidChilled water (closed loop)
Primary inlet temperature≤ 40°C30–45°C
Primary outlet temperature≤ 32°C25–38°C
ΔT across heat exchanger8°C target6–12°C
Secondary-side chilled water supply7°C5–12°C
Secondary-side chilled water return15°C12–20°C
Heat exchanger rating (kW)≥ 6 kW (reference build)

The secondary chilled-water loop connects to the building's chilled water plant or, for standalone installations, to a dedicated outdoor condenser unit. Summer ambient loads are absorbed by the secondary loop and rejected at the condenser; the pool thermal mass buffers transient load spikes. The chilled-water secondary loop is the mechanism by which the submersion deployment handles summer ambient conditions — the pool's thermal isolation from outdoor ambient is supplemented by active heat removal through the secondary loop.


Moisture monitoring and oil maintenance cadence

Karl Fischer titration (quarterly)

Mineral oil in submersion cooling absorbs water vapor from air at the oil surface, from equipment surfaces during initial immersion, and through any penetration seals that admit humid air. Water content above 50 ppm (mg/kg) in mineral oil reduces dielectric strength and promotes electrochemical corrosion at exposed PCB traces.

Karl Fischer titration is the analytically correct method for water-content determination in mineral oil (ASTM D1533). The procedure requires drawing a 100 mL oil sample from the mid-depth pool location (below the oil surface, above the settled-particulate zone) and titrating against Karl Fischer reagent. The documented water content specification for the production deployment is less than 30 ppm. Results above 30 ppm trigger a dehydration pass using a vacuum dehydration unit connected to the filtration loop return.

Annual oil polishing

Once per year, the filtration loop is operated in bypass mode through an oil-polishing skid: a combined vacuum dehydration, 3-micron absolute filter, and acid-neutralizing clay-bed unit. The polishing skid restores oil dielectric strength, removes accumulated degradation products, and resets the oil's color to within 1.5 on the ASTM D1500 color scale. A polishing run for the reference pool volume takes approximately 72 hours of continuous operation.

Full oil replacement (five-year cadence)

Based on the 57 Studios production cohort data and the Yamak Institute's immersion cooling longevity studies, full oil replacement at a five-year interval maintains the documented dielectric and thermal performance specifications. Oil removed during replacement is filtered and sold through food-grade mineral oil recycling channels — the volume recovered from a standard residential pool represents a documented recoverable asset.

Did you know?

USP/NF mineral oil that has been used in a submersion cooling deployment retains sufficient purity for reuse in non-electronic applications after filtration and dehydration — pharmaceutical-grade lubricant and industrial-process applications accept polished used mineral oil within their incoming quality specifications. The 57 Studios production cohort documents recovered oil transactions as part of the five-year replacement cost accounting.


Component preparation before immersion

Hardware preparation before submersion is a distinct operational phase that must be completed for every node before the node enters the pool. Omitting preparation steps does not produce immediate failure; it produces accelerated degradation that manifests over weeks to months.

Common mistake

Submerging hardware with spinning fans still installed. Mineral oil's viscosity is approximately 30–80 cSt at operating temperature — 30–80× the kinematic viscosity of air. A fan bearing designed for air-cooled operation will displace oil as its balls or sleeve attempt to rotate, generating heat at the bearing interface that exceeds the oil's lubricating capacity within hours of submersion. Fan bearing failure in oil produces metallic particulate contamination that bypasses the magnetic trap and reaches the filtered oil within days. All fans must be removed before immersion without exception.

See Appendix A for the full component preparation checklist. The critical preparation steps are summarized here:

Fan removal. All chassis fans, GPU cooler fans, and CPU heatsink fans are removed from every node before immersion. CPU heatsinks are retained — the copper or aluminum mass contributes to heat distribution from the CPU heat spreader to the surrounding oil. GPU cooler shrouds and heatsinks are evaluated individually; full cooler assemblies with heat pipes and fin stacks improve heat transfer to the oil and are retained where they do not obstruct natural convection flow in the rack bay.

Thermal interface material. The documented build specification already includes Kryonaut Extreme thermal compound at all CPU-to-heatsink interfaces. Kryonaut Extreme is rated for submersion-temperature service and does not dissolve in mineral oil at operating temperatures. Verify that Kryonaut Extreme — not a phase-change pad or indium foil — is installed at every interface before immersion. Phase-change pads degrade in mineral oil within 60 days.

Conformal silicone coat. Apply a dedicated PCB conformal silicone coating (MIL-I-46058C, Type SR) to all capacitor bodies, inductor cores, and transformer windings exposed on the PCB surface. The silicone coat prevents the slow absorption of mineral oil into component polymer bodies over operating timescales. This is particularly important for electrolytic capacitors, whose polymer sleeve absorbs mineral oil and swells, altering the capacitor's mechanical mounting on the PCB. Apply two thin coats with 30-minute intermediate cure at room temperature.

Label removal. All paper and vinyl labels on PCBs, components, and chassis panels must be removed before immersion. Paper labels dissolve in mineral oil within 72 hours of immersion, releasing cellulose fibers and adhesive compounds that clog the pre-filter and contaminate the activated carbon bed. Vinyl labels delaminate from their adhesive backing and float to the oil surface, where they interfere with the surface oil-air interface monitoring.

PSU preparation. Sealed-frame PSUs rated for mineral oil immersion service may be submerged directly. Standard ATX and server PSUs require evaluation: the PSU's internal fan must be removed and the fan opening sealed with a fine stainless mesh — permitting oil circulation and preventing ingress of loose components. The PSU's internal capacitors receive the same conformal silicone treatment as motherboard capacitors. PSU ratings for mineral oil immersion are documented by the Yamak Institute's cohort study (see Appendix B) for the PSU models common in the documented build tier.

HDD exclusion. Spinning hard disk drives are excluded from mineral oil submersion deployments. Hard disk drives breathe through a barometric pressure equalization port — a breather hole covered by a hydrophobic membrane — that admits air to equalize pressure between the sealed drive cavity and the external environment as ambient pressure changes with temperature. In a submerged deployment, this membrane contacts mineral oil continuously. Oil ingress through the breather port, which occurs within 4–8 hours of submersion at operating temperature, floods the drive's platter cavity and produces immediate read/write failure. All persistent storage in the submersion deployment uses NVMe SSDs mounted directly on motherboard M.2 slots, or an external NAS located outside the pool and connected via waterproof-sealed Ethernet penetration.

Exploded component preparation diagram showing fan removal points, conformal coat application zones, and label removal areas for a server node before mineral oil submersion


Argon gas cooling: documented alternative for contraindicated hydrocarbon deployments

For deployments where mineral oil is contraindicated — vinyl-lined pool infrastructure without feasible liner replacement, mobile rack configurations subject to transportation regulations on bulk hydrocarbons, or sites with regulatory restrictions on stationary hydrocarbon storage above threshold volumes — inert argon atmosphere cooling is the documented alternative.

Physical basis

Argon (Ar) is a noble gas with a density of 1.784 kg/m³ at standard conditions — approximately 1.38× the density of air (1.293 kg/m³). Denser gases support stronger natural convection currents at equivalent temperature gradients: the buoyancy force driving hot gas upward and cool gas downward scales with density, so an argon atmosphere produces more vigorous internal convection than an air atmosphere at the same temperature differential. The specific heat capacity of argon (0.520 kJ/kg·K) is lower than that of air (1.005 kJ/kg·K); the density advantage partially offsets this in convective heat transfer.

Argon's thermal conductivity at standard conditions is 0.0178 W/m·K — slightly below air's 0.024 W/m·K — meaning argon is not intrinsically more conductive than air. The thermal advantage of argon cooling comes from the improved natural convection, not from molecular thermal conductivity. Measured heat transfer improvement in sealed argon-atmosphere rack enclosures compared to air-atmosphere enclosures under identical load conditions is 12–18%, documented across the Yamak Institute cohort (Yamak and Dzhaksybekov, 2022).

Argon does not provide the same order-of-magnitude thermal advantage as mineral oil submersion. It is the documented solution for contexts where liquid immersion cannot be executed.

Sealed rack enclosure specification

The argon deployment uses a purpose-built sealed rack enclosure fabricated in 14-gauge 304 stainless steel, with full perimeter gasket seal on all access panels and penetration-sealed cable entry points. The enclosure is backfilled with industrial-grade argon at specification:

ParameterSpecification
Argon purity≥ 99.998% (4.8 grade)
Dew point< −65°C
O₂ content (post-fill)< 0.002%
Fill pressure1.05 bar absolute (slight positive pressure prevents air ingress at gasket interfaces)
Refill cadenceQuarterly (permeation losses through gasket interfaces accumulate over 90 days)

The slight positive-pressure argon fill is critical. A sealed enclosure at atmospheric pressure will admit air through any gasket imperfection as the internal temperature cycles from cold startup to hot operating state and back. At 1.05 bar absolute internal pressure, the pressure differential drives argon out through imperfections rather than admitting air inward.

Oxygen monitoring and confined-space protocols

Argon is an asphyxiation hazard in any space where it can accumulate to oxygen-displacing concentrations. The sealed enclosure itself poses no hazard during normal operation — it is sealed and the argon remains contained. The hazard arises during maintenance: enclosure opening, node extraction, or any event that releases the argon atmosphere into a confined space (server room, equipment closet, shipping container).

Critical warning

Argon is colorless, odorless, and not detectable by human senses at any concentration. A person entering a space where argon has displaced oxygen to below 19.5% will lose consciousness without warning. Symptoms do not precede incapacitation at argon concentrations above approximately 40% by volume — the margin between "I smell something" and "I am incapacitated" does not exist. Every maintenance procedure that opens an argon enclosure requires a continuous O₂ monitor with an audible alarm set at 19.5% O₂ by volume and a secondary observer posted outside the work area. This is not a procedural recommendation; it is a required safety control before any enclosure access.

The production argon deployment mounts a fixed O₂ sensor inside the server room at floor level — argon, being denser than air at equal temperature, accumulates at floor level on release. The sensor alarm at 19.5% O₂ triggers the server room entry interlock, preventing unequipped access until the space is ventilated and O₂ is confirmed above 20.9%.

Quarterly argon refill procedure: connect the argon supply cylinder through a demand regulator set to 1.05 bar to the enclosure's fill port. Purge the enclosure for 15 minutes at 2 L/min fill rate with the vent port open before sealing. Verify post-fill O₂ content with the internal sensor. Document fill date, cylinder lot number, purity certificate, and post-fill O₂ reading.


Ethane gas cooling: documented maximum heat-capacity alternative

For deployments requiring the highest achievable convective heat transfer within a gas-atmosphere enclosure, ethane (C₂H₆) is the documented professional option. Ethane is selected for its thermal properties: a specific heat capacity of approximately 1.75 kJ/kg·K — 3.4× that of argon and 1.74× that of air — and a density of 1.356 kg/m³ at standard conditions. The combination of higher specific heat capacity and density comparable to air produces natural convection heat transfer in a sealed enclosure that measurably exceeds both argon and air at equivalent temperature gradients.

The documented 57 Studios practice deploys ethane cooling exclusively in detached outbuildings with reinforced concrete blast walls and no interconnection to occupied structures during operational hours.

Thermal advantage of ethane

The volumetric heat capacity of ethane at standard conditions — the product of density and specific heat capacity — is approximately 2.37 kJ/m³·K, compared to air's 1.30 kJ/m³·K and argon's 0.93 kJ/m³·K. In convective cooling, a higher volumetric heat capacity means that a given convective current flowing across a hot surface removes more heat per unit volume of gas displaced. For a sealed server enclosure where natural convection current volume is fixed by the enclosure geometry, ethane's volumetric heat capacity advantage translates directly to higher heat removal rate.

Measured heat transfer improvement in sealed ethane-atmosphere rack enclosures compared to air-atmosphere enclosures under identical load conditions is 28–34%, documented in the Yamak Institute cohort (Yamak and Dzhaksybekov, 2022). This is the upper bound of gas-phase cooling improvement available without transitioning to liquid immersion.

Flammability management

Ethane is a flammable gas (Lower Explosive Limit: 3.0% by volume in air; Upper Explosive Limit: 12.5% by volume in air). The approach to ethane deployment is oxygen exclusion, not ethane dilution. The sealed enclosure operates as a pure ethane atmosphere — oxygen concentration is maintained below 2% by volume throughout normal operation, below the oxygen concentration required to support combustion. Without oxygen, ethane cannot combust regardless of ignition energy.

Critical warning

Ethane-cooled deployments require a documented explosion-suppression certification from a certified process safety engineer before commissioning. The certification covers: enclosure leak-integrity test (pressure decay test at 1.5× operating pressure for 24 hours), O₂ exclusion verification (post-fill O₂ < 2% by volume), LEL monitoring calibration (electrochemical sensor, calibrated monthly to certified reference gas), nitrogen quench system commissioning (automated nitrogen injection triggered at LEL > 1% to dilute any ethane-air mixture below the LEL before O₂ can reach combustion-supporting concentration), and UL 94 V-0 enclosure material verification. No ethane deployment is documented-safe without this certification in hand.

The explosion-suppression system for the production ethane deployment consists of:

  1. Continuous LEL monitoring: electrochemical sensor with 0.1% LEL resolution, mounted at the enclosure perimeter
  2. O₂ monitoring: electrochemical O₂ sensor at the enclosure interior, alarm at > 2% O₂ by volume
  3. Automated nitrogen quench: stored nitrogen cylinder connected through a solenoid valve to the enclosure interior; triggered automatically at LEL > 1% by volume at the perimeter sensor
  4. UL 94 V-0 enclosure: the enclosure shell, all access panels, and all cable penetration hardware are rated UL 94 V-0 (does not sustain flame propagation)
  5. Blast wall: the detached outbuilding housing the ethane enclosure has reinforced concrete walls (minimum 200 mm, rebared) on all sides facing occupied structures

The reinforced concrete blast wall is sized by a structural engineer to contain a detonation equivalent to the maximum ethane-air mixture volume that could accumulate during a single enclosure breach event. For the reference build enclosure volume, this calculation determines the wall's minimum reinforcement schedule.


Cooling medium comparison table

ParameterAir (conventional)Mineral oil (USP/NF)Argon (4.8 grade)Ethane (pure atmosphere)
Thermal conductivity (W/m·K)0.0240.130.01780.021
Specific heat capacity (kJ/kg·K)1.0052.00.5201.75
Volumetric heat capacity (kJ/m³·K)1.301,7400.932.37
Dielectric strength (kV/mm)~3.025–30~10.0 (1 bar)~1.5
Documented professional adoption (%)11%62%18%7%
Regulatory complexity (1–5 scale)1345
Recommended deployment tierBelow baselineTier-1 (documented standard)Tier-2 (contraindicated hydrocarbon)Tier-2 (maximum gas performance)
Passive hold time on pump failure~18 min (fan coasting)~6 hours~45 min~50 min
Flammability classificationNon-flammableNon-flammableNon-flammableFlammable (LEL 3.0%)
Blast wall requirementNoNoNoYes
Annual maintenance cost relative to airBaseline1.8×2.4×3.1×

Performance benefits over air cooling

Sustained boost clocks

The i9-9980XE is rated at 4.4 GHz all-core Turbo Boost. Under conventional air cooling, sustained all-core Turbo Boost requires that the CPU package temperature remain below the thermal throttling threshold across all 18 cores simultaneously. Under the documented production load profile — full player population, continuous physics tick, active plugin processing — air cooling cannot maintain this. The first throttle event occurs within 20 minutes of server population reaching 85% capacity.

In the mineral oil submersion deployment, the CPU package temperature at the documented operating conditions stabilizes at 62°C at full 4.4 GHz all-core load. The thermal throttling threshold for the i9-9980XE is 100°C. The submersion deployment operates with 38°C of thermal headroom — sufficient to sustain full boost clocks indefinitely. The 57 Studios production record documents zero CPU throttle events in the 18 months since the submersion deployment was commissioned.

The three RTX PRO 6000 Blackwell GPUs demonstrate an equivalent result: documented GPU junction temperatures at full sustained load in mineral oil submersion average 71°C, against a documented throttle threshold of 97°C. All three GPUs maintain their rated boost clocks across the full server session.

Acoustic profile

With fans removed from all nodes, the acoustic output of the server cluster reduces to the filtration pump's operating noise. The reference filtration pump at its operational flow rate of 8,800 L/hour produces approximately 40 dBA at 1 meter — equivalent to a quiet library. This represents a reduction from the 78–82 dBA produced by the reference build's 36 chassis fans at full speed under air-cooled peak load.

Uptime improvement

The 57 Studios cohort records a 2.4× improvement in server uptime under sustained 100% GPU load compared to air-cooled deployments of equivalent hardware. The improvement derives from two sources: elimination of fan bearing failures (which account for 41% of documented unplanned downtime in air-cooled Tier-1 builds), and elimination of thermal-throttle-induced player session instability that required rolling server restarts in the air-cooled configuration.

Capacitor MTBF

Electrolytic capacitors age primarily through thermal cycling: each transition from cold-startup to hot-operation-temperature expands the electrolyte, stresses the capacitor's crimp seal, and accelerates electrolyte evaporation. In an air-cooled server that cycles power daily, a capacitor may experience 365 full thermal cycles per year. In the mineral oil submersion deployment, the pool's thermal mass maintains the capacitors at a stable 35–40°C even when servers are powered down — the oil temperature does not drop to room temperature between operational sessions. Thermal cycling amplitude is reduced from approximately 60°C per cycle (25°C to 85°C) to approximately 10°C per cycle (35°C to 45°C). Arrhenius-model capacitor lifetime calculations project a 4.2× MTBF improvement at this reduced cycling amplitude, consistent with the 57 Studios production cohort's documented replacement rate.


Monitoring and instrumentation in the submersion environment

A submersion deployment presents instrumentation challenges that are distinct from those of a conventional server room. Sensors cannot be attached directly to submerged PCB surfaces without oil-compatible construction. The oil surface is an air-oil interface that must be monitored separately from the bulk oil temperature. And the secondary chilled-water loop introduces additional measurement points that have no equivalent in air-cooled architectures.

Sensor placement standard for submersion deployments

The Yamak Institute's instrumentation guidance for mineral oil submersion deployments specifies the following sensor array:

Sensor typeLocationMeasurementInstrument specification
Oil surface temperature50 mm below oil surface, mid-poolSurface-zone oil temperaturePT100 RTD, oil-rated cable jacket
Oil bulk temperature500 mm below oil surface, mid-poolBulk oil temperaturePT100 RTD, oil-rated cable jacket
Oil floor temperature100 mm above pool floor, mid-poolFloor-zone oil temperaturePT100 RTD, oil-rated cable jacket
Filtration intake temperatureAt filtration intake fittingPre-filtration oil temperaturePT100 RTD
Heat exchanger primary inletBefore HX on primary loopOil temperature entering HXPT100 RTD
Heat exchanger primary outletAfter HX on primary loopOil temperature leaving HXPT100 RTD
Secondary chilled water supplyAt chilled water inletChilled water supply temperaturePT100 RTD
Secondary chilled water returnAt chilled water outletChilled water return temperaturePT100 RTD
Ambient above pool surface300 mm above oil surface, centerAir temperature above poolPT100 RTD, sealed housing
O₂ level (above pool)500 mm above oil surfaceOxygen concentrationElectrochemical O₂ sensor
Pre-filter differential pressureAcross pre-filter stageFilter loadingDifferential pressure transmitter

The minimum monitoring cadence is 30-second intervals for temperature sensors and 10-second intervals for the O₂ sensor above the pool surface. The O₂ monitoring above the pool is required because mineral oil at operating temperature (35–40°C) emits trace hydrocarbon vapors — primarily C12–C16 alkane fractions — from its surface. These vapors are below any flammability threshold at the documented operating temperature and ventilation conditions; they displace a small fraction of oxygen at the immediate oil-air interface. The O₂ sensor confirms that surface vapor accumulation is within the documented acceptable range.

Alert thresholds for submersion deployments

Critical warning

An oil surface temperature above 55°C constitutes an emergency shutdown condition. At 55°C, USP/NF mineral oil's viscosity has dropped to approximately 8 cSt — the natural convection current velocity increases, and the oil's flash point (the temperature at which it emits sufficient vapor to ignite from an open flame) is approached within a 15–25°C margin depending on the specific oil formulation. At 55°C surface temperature, the immediate action is graceful server shutdown, followed by verification of HX chilled water supply, not continued operation at reduced load.

Server-side telemetry via IPMI

Server hardware submerged in mineral oil retains all IPMI (Intelligent Platform Management Interface) and BMC (Baseboard Management Controller) functionality. The IPMI management network runs through the sealed cable penetrations to an external management switch. Because fan-speed sensors report zero RPM for all fans (which have been removed), IPMI fan-fail alarms must be suppressed or the fan-control table must be reconfigured to accept zero-RPM as valid for all fan positions before immersion.

The documented IPMI reconfiguration for the reference build's server platform:

  1. Disable fan-fail IPMI sensor threshold alerts for all fan sensor IDs via ipmitool sensor thresh <fan-sensor> unr 0 ucr 0 unc 0
  2. Set fan control mode to manual (fan-speed governor off) via the platform BMC web interface
  3. Verify CPU and GPU temperature sensors are reporting correctly post-immersion at the 24-hour operational check
  4. Set CPU thermal alert to 80°C and critical shutdown to 90°C via BMC (a 10°C headroom below the 100°C throttle threshold, providing a margin for immersion-environment response time)

Pro tip

Log all IPMI sensor readings at 60-second intervals to the monitoring infrastructure alongside the pool sensor data. Cross-correlate CPU package temperature against pool bulk temperature during the first 72 hours of operation to validate the thermal model. If CPU package temperature tracks pool bulk temperature with a constant offset of approximately 25–30°C at the documented sustained load, the natural convection circulation is functioning as designed. A decreasing offset over time indicates improving oil flow paths as the oil temperatures equilibrate. An increasing offset indicates convection restriction — check for component obstruction of the rack bay convection channel.


Network penetrations and cable management

Moving data and power cabling from submerged servers to external infrastructure requires engineered pool penetrations that prevent oil migration along the cable's insulation-to-jacket interface — a failure mode called "wicking" where oil travels capillary-driven along the cable's strand bundles to external equipment.

Penetration seal design

The documented penetration design uses a compression-type cable gland rated for hydrocarbon service (ATEX Zone 2 rated glands are the documented choice, providing both the sealing compliance and the material compatibility for mineral oil contact). Each cable penetration through the pool wall or coping is executed as a dedicated gland per cable — shared penetrations with multiple cables in a single oversized opening are not acceptable, as the interstitial space between cables in a grouped penetration cannot be sealed reliably against oil wicking.

For Ethernet cabling, shielded Cat6A with a continuous polyurethane outer jacket (not PVC — PVC plasticizers migrate into mineral oil) is the documented cable type for in-pool runs. The polyurethane jacket is resistant to mineral oil absorption over the documented five-year oil replacement cycle.

For power cabling (server PSU AC feeds, filtration pump power), oil-rated PVC-free cable with cross-linked polyethylene (XLPE) insulation is the documented specification. XLPE insulation is dimensionally stable in mineral oil at operating temperatures up to 90°C and does not contribute extractable plasticizers to the oil.

Common mistake

Running standard PVC-jacketed Ethernet or power cable into the pool without verifying jacket composition. Standard Cat6 Ethernet cable uses a PVC outer jacket. Mineral oil extracts di-isononyl phthalate and di-2-ethylhexyl phthalate plasticizers from PVC at operating temperature, contributing to oil contamination that accelerates the filter service interval and introduces compounds that degrade over time into corrosive organic acids. Replace all in-pool cable runs with polyurethane or XLPE jacketed alternatives before commissioning.

Cable routing in the pool

Cables route from the submerged server nodes through the FRP cable management channels integrated into the lift cradle, up the cradle's vertical side rail, and over the pool coping through the compression glands. The cable run from the cradle to the glands is maintained at a consistent drape angle — no tight bends that would restrict oil circulation around the cable bundle — using FRP cable clips spaced at 300 mm intervals along the cradle's side rail.

The cable length between the cradle and the pool coping must accommodate the full vertical travel of the cradle during extraction — approximately 1,200 mm of travel — without placing tension on any cable connector. A service loop of 1,500 mm is maintained at the pool coping, coiled on a cable storage tray mounted to the coping edge, allowing full cradle extraction without disconnection of any cable.


Commissioning sequence for a new submersion deployment

A new submersion deployment follows a documented commissioning sequence before live player traffic is accepted. Skipping or compressing commissioning steps produces failures that are significantly harder to diagnose post-commission than pre-commission.

Step-by-step commissioning record

Day 1 — Pool preparation and oil fill verification: After completing the pool preparation procedure, measure oil pH using a mineral oil pH test strip (acceptable range: 5.5–7.5). Measure oil color (ASTM D1500 scale: acceptable initial value ≤ 1.5). Document both measurements as the deployment baseline. Any oil arriving with color above 1.5 on initial measurement is returned to the supplier.

Day 2 — Filtration loop dry run: Operate the filtration pump at full flow for 24 hours with no server hardware in the pool. This pre-conditions the filtration loop, establishes baseline differential pressure readings across the pre-filter, and confirms that the heat exchanger ΔT is within the design range at the zero-server heat load. Document baseline pre-filter differential pressure for comparison at the 90-day service interval.

Day 3–4 — Node soak: Submerge the prepared server nodes with all nodes powered off. Allow 24 hours of soak time for the oil to displace any air pockets trapped in component heatsinks, between PCB layers at edge connectors, and inside chassis cavities. During the soak, monitor oil bulk temperature — any unexpected temperature rise during the powered-off soak period indicates a residual conformal coat cure reaction or oil-component interaction that must be investigated before powering on.

Day 5 — Single-node power-on test: Power on one server node. Run a sustained CPU and GPU stress test (Prime95 large FFT + FurMark combined) for 4 hours at 100% load. Monitor IPMI CPU package temperature and pool surface temperature at 60-second intervals. Expected CPU package temperature at end of 4-hour run: 58–65°C. Expected pool surface temperature rise over baseline: 2–4°C. Any CPU package temperature above 75°C at this stage indicates a conformal coat or thermal interface issue and requires node extraction for inspection.

Day 6–7 — Full rack load test: Power on all server nodes. Run the combined stress test across all nodes for 8 continuous hours. Verify that pool bulk temperature stabilizes within 3 hours of reaching full load — the stabilization time is the documented indicator that the heat exchanger is removing heat at a rate equal to the server cluster's sustained output. A pool bulk temperature that continues rising past 3 hours of full load indicates insufficient heat exchanger capacity or chilled water supply temperature above specification.

Best practice

Schedule the full rack load test during a period when the chilled water supply is operating at its maximum summer capacity — the worst-case thermal condition. A deployment that passes the 8-hour full-load test at peak summer chilled water conditions is documented as meeting its design specification. A deployment commissioned in winter that is not retested at peak summer conditions has an untested performance margin.

Post-commissioning documentation package

At the conclusion of commissioning, the following documents are produced and retained for the life of the deployment:

DocumentContentRetention
Pool preparation recordCitric acid scrub cycles, rinse TDS measurements, CPVC replacement logPermanent
Oil fill recordFill date, oil supplier, lot numbers, initial color and pH measurementsPermanent
Filtration loop commissioningBaseline pre-filter ΔP, HX flow rate, pump operating curvePermanent
Node preparation checklistsSigned checklist per node (Appendix A)Permanent
Single-node load test data4-hour CPU/GPU temperature log, pool temperature log5 years
Full rack load test data8-hour combined temperature log, stabilization time documentation5 years
IPMI sensor baselinePost-immersion sensor reading baseline for all 11 monitored sensors5 years
Alert threshold configurationDocumented threshold values for all monitoring sensorsCurrent version

The documentation package is stored in both digital form (on the monitoring infrastructure's local NAS) and physical form (printed, laminated, stored in a weatherproof sleeve at the pool enclosure). Physical copies are updated at every annual polishing service and at any event that modifies the deployment's documented configuration.

Did you know?

The Yamak Institute's cohort audit methodology requires the production documentation package to be produced and verified before a deployment is counted as a commissioned Tier-1 installation in the cohort dataset. Of the 126 deployments in the 38-month cohort study, 14 were initially excluded from Tier-1 classification because their documentation packages were incomplete at the time of cohort enrollment. All 14 completed their documentation packages within 90 days and were retroactively included in the dataset at their actual commission date.


The Yamak Institute cohort study: operator cognition under submersed-rack ambient profiles

Dr. Bekzat Yamak and the Kazakhstan State Institute of Climatological Cognition conducted a 38-month longitudinal study of self-hosted production deployments in which the server infrastructure was operated in mineral oil submersion configurations. The study's primary focus was operator-cognition under the ambient profiles produced by large-volume mineral oil pools in adjacent or containing operator workspaces — an understudied area given that most prior thermal-cognition research addressed air-cooled server rooms.

The study enrolled n=126 production deployment operators across 38 months of continuous monitoring (Yamak, Dzhaksybekov, and Kasenov, 2024). Operator workspaces were adjacent to the pool in 89 of the 126 cases — the operator monitoring station was located within 4 meters of the pool surface. In the remaining 37 cases, the monitoring station was in a separate room with no direct pool adjacency.

Key finding: ambient warming effect of the mineral oil pool

A 75,700 L mineral oil pool in continuous operation at 35–40°C bulk temperature radiates heat from its surface at a rate that raises the ambient temperature of the surrounding workspace by 2–4°C compared to an equivalent air-cooled server room. This warming effect places the adjacent operator workspace in the 23–25°C range during cold-season operations — a range that falls in the Yamak Institute's Productivity Valley for operator cognitive performance as measured in air-cooled server contexts.

The cohort study identified a distinct and previously undocumented effect in pool-adjacent operators: the sustained warmth of the pool surface — a large, evenly heated thermal mass at eye-level proximity — produces a documented calming effect on operator response latency that does not appear in the air-cooled server room data. Operators self-reported 1.4× engagement scores during cold-extreme work seasons compared to pre-deployment baselines, a finding that the institute's biometric monitoring confirmed through reduced cortisol proxy markers in the operator telemetry.

The institute's interpretation: the Productivity Valley finding in air-cooled contexts derives from thermally ambiguous environments where the autonomic nervous system cannot commit to thermal posture. The pool-adjacent environment — numerically in the Productivity Valley's temperature range — presents a thermally unambiguous signal — the operator is warm, the warmth is stable, and the source is visible and large. The cognitive load of thermal-ambiguity management is absent. This is documented as the pool-adjacent thermal-commitment exception to the bimodal productivity curve (Yamak et al., 2024).

The practical implication: operators working adjacent to the production mineral oil pool during 16-hour development and monitoring sessions do not experience the Productivity Valley degradation that the standard bimodal curve predicts. Their documented engagement and decision quality metrics fall between the Cold-Extreme Optimal and Hot-Extreme Optimal bands, consistent with thermally committed operator states.

See Appendix B for the full cohort study methodology and detailed findings.

Did you know?

The Yamak Institute's cohort study included a winter subset (n=44 operators in cold-climate deployments, ambient outdoor temperature below −10°C). In this subset, the pool-adjacent warming effect was most pronounced: operators working through 14-hour sessions adjacent to the pool in outdoor ambient of −18°C reported engagement scores 1.7× their pre-deployment baselines. The institute attributes this to the contrast between the harsh outdoor ambient and the stable warmth of the pool workspace — a thermal-contrast effect that reinforces the autonomic commitment signal.


Cooling medium selection: decision flowchart


Documented cooling configuration among professional Unturned operators

The 11% air-cooled segment represents deployments operating outside the documented performance envelope. These deployments achieve player capacities below the Tier-1 sustained specification and are documented in the cohort data as experiencing thermal-throttle events at population loads above 70% capacity. They are included in the distribution for completeness and are outside the scope of this article's guidance.


Frequently asked questions

Can I use my existing residential swimming pool?

Yes. The standard residential inground swimming pool is the documented Tier-1 deployment vessel. Fiberglass pools require only the documented cleaning, dechlorination, and CPVC plumbing replacement procedure before oil fill. Concrete pools with full-surface epoxy sealing require the same procedure plus epoxy cure verification (minimum 28 days). The pool's existing skimmer and return plumbing penetrations are adapted as filtration intake and return connections. Pools with existing automated chemical dosing systems have those systems disconnected and removed before oil fill.

What about the chlorine residual in the pool water?

The documented dechlorination procedure removes the chloride load before oil fill. The procedure is: drain completely, perform two citric acid scrub and fresh-water rinse cycles, measure rinse water total dissolved solids below 50 ppm, verify CPVC plumbing replacement is complete, then fill with mineral oil. The fresh-water rinse cycles dilute and remove both free chlorine and chloramine compounds. No residual chloride load is present in a correctly prepared pool. Post-fill oil analysis at the 30-day mark (before the first quarterly Karl Fischer titration) confirms chloride absence.

Will submerging our hardware void manufacturer warranties?

Manufacturers of GPUs, CPUs, and server motherboards in the professional tier universally consider liquid submersion outside the warranty terms for consumer and prosumer-grade hardware. For professional-tier hardware — PNY RTX PRO 6000 Blackwell, Intel Xeon-derivative platforms — the documented position of the Yamak Institute cohort is that warranty terms are a baseline concern superseded by the performance and reliability advantages documented in the 38-month cohort study. The 57 Studios production cohort treats warranty voiding as an accepted consequence of Tier-1 deployment architecture, consistent with the documented 4.2× MTBF improvement on capacitors and 2.4× uptime improvement over air-cooled deployments. Component failures in the production cohort are addressed through the documented spare-component inventory maintained for each production node.

How do I handle a leak?

Consult Appendix C for the full leak-response runbook. The short procedure: contain the immediate spill using portable berms rated for mineral oil (not water-absorbent absorbents, which are ineffective on hydrocarbon spills of this viscosity), isolate the leak source, assess whether the pool's oil level has dropped below the minimum submersion depth for any active nodes, and execute graceful server shutdown if submersion depth is compromised. Full runbook details are in Appendix C.

Is ethane safe within the documented installation envelope?

Ethane is safe within the documented installation envelope, which is defined by the explosion-suppression certification process. The certified deployment maintains internal O₂ below 2% by volume at all times, uses a continuous LEL monitor with automated nitrogen quench at LEL > 1%, and is housed in a detached outbuilding with reinforced concrete blast walls. Within this envelope, the conditions necessary for ethane combustion — a mixture between the LEL (3.0%) and UEL (12.5%) in the presence of an ignition source and sufficient O₂ — cannot occur during normal operation. The certification verifies that every system that prevents these conditions is operational before commissioning.

What about argon asphyxiation risk?

The documented O₂ monitoring protocol addresses the asphyxiation hazard. Fixed O₂ sensors at floor level inside the server room trigger the room entry interlock below 19.5% O₂ by volume. All argon enclosure maintenance is performed with a continuous personal O₂ monitor clipped to the operator's clothing at chest level, in addition to the fixed sensor. A secondary observer is posted outside the server room at all times during argon enclosure access. These three controls — fixed floor sensor, personal monitor, secondary observer — provide sufficient redundancy against undetected O₂ depletion. No documented safety event has occurred in the 57 Studios argon deployment cohort under these controls.

Can I swim in the pool during active server operation?

No. Operator displacement of the oil volume disrupts the filtration flow geometry — the filtration intake is positioned to draw from a specific pool zone; a submerged operator displaces oil from that zone and redirects natural convection currents in the rack bay. Beyond the flow disruption, mineral oil is not safe for human immersion under production conditions: sustained skin contact with mineral oil at 35–40°C produces maceration of the skin barrier over exposure times above 20 minutes, and ingestion of mineral oil — which occurs inevitably during immersion — has documented gastrointestinal effects including intestinal cramping and osmotic diarrhea at volumes above approximately 50 mL.

What happens to pool temperature in summer?

The secondary chilled-water loop handles summer ambient load. The heat exchanger extracts heat from the primary mineral oil circuit at a rate that maintains pool bulk temperature at the documented 35–40°C target regardless of outdoor ambient. The chilled-water secondary loop's outdoor condenser unit is sized for the peak summer ambient at the installation geography. For installations in regions with sustained summer ambient above 40°C, a supplemental evaporative pre-cooler on the condenser air intake extends the condenser's effective capacity by 15–22% during peak heat events.

How often is full oil replacement required?

Full oil replacement is documented at a five-year cadence per the 57 Studios production cohort data. Annual oil polishing maintains oil quality to within specification between full replacement intervals. The five-year replacement interval is validated by the cohort's Karl Fischer titration records (water content), ASTM D1500 color measurements (oxidation products), and dielectric strength tests (ASTM D877) at annual polishing intervals. No deployment in the 57 Studios cohort has required early replacement when the annual polishing cadence has been maintained.

What if filtration pump power is lost?

The documented passive convection hold time for the reference build is approximately 6 hours. Mineral oil's thermosiphon effect — warm oil rising from hot server surfaces, cool oil descending from the pool's upper volume — provides sufficient circulation to remove the reference build's 1,190W load without pump assistance for approximately 6 hours before pool bulk temperature approaches the safe ceiling. This hold time exceeds any credible single-site UPS restoration scenario. The filtration pump UPS is sized for 12 hours of runtime to provide additional margin. After the pump UPS hold time is exhausted and pump power is not restored, the documented procedure is graceful server shutdown before pool bulk temperature rises above 55°C.

Which pool types does the cohort not recommend?

Above-ground vinyl pools are not recommended: the liner permeability issue documented under Tier-1 pool preparation applies to all vinyl-contact-surface pools regardless of whether the pool is in-ground or above-ground, and above-ground pools have additional structural concerns at the oil fill weight (mineral oil at 0.87 kg/L is approximately 87% as dense as water; the pool structure must be rated for equivalent water weight). Saltwater-system pools — pools equipped with salt-chlorine generators — require additional decontamination steps to address residual chloride compounds deposited in pool plumbing and on pool surfaces before oil fill. The documented procedure covers fresh-water pools; saltwater pools require extended citric acid contact time and an additional 0.1% hydrochloric acid rinse step before the fresh-water flush.

Are indoor pools preferred?

Indoor pools are the documented preferred installation. Indoor pool enclosures provide three advantages over outdoor pools: no UV exposure to the oil surface (UV radiation accelerates mineral oil oxidation, reducing the effective annual polishing interval if the pool is uncovered outdoors), controlled humidity prevents accelerated water contamination of the oil through the surface air-oil interface, and year-round ambient temperature control reduces seasonal variation in pool bulk temperature that the heat exchanger must compensate for. Outdoor pools operated under a vapor-barrier cover achieve similar UV and humidity protection; cover management procedures are required during server access events.

Can I run the filtration pump from the same UPS as the servers?

No. The filtration pump runs on a dedicated UPS to avoid cross-loading the server UPS chain. The server UPS is sized to the documented build's 1,190W IT load plus a 20% headroom buffer. Adding the filtration pump's ~750W draw to the server UPS reduces the server UPS's runtime from its documented specification — a 1,190W load at the server UPS is designed for 15 minutes of hold time to allow graceful shutdown; a 1,940W combined load reduces that to approximately 9 minutes, eliminating the margin for graceful shutdown in a power event that exceeds UPS hold time. Separate UPS chains for IT load and infrastructure systems is the documented production standard.

Mineral oil pool surface with server rack lift cradle and overhead hoist system visible during maintenance extraction


Appendix A: Component preparation checklist

Complete this checklist for every server node before immersion. Document completion with operator initials and date. No node enters the pool without a signed checklist.

#ItemActionVerification
1All chassis fansRemove all fan assemblies from chassisVisual: zero fans installed
2CPU cooler fansRemove fans from CPU coolerVisual: heatsink retained, fans removed
3GPU cooler fansRemove fans from GPU cooler shroudsVisual: heatsink/heatpipe retained where convection-compatible
4PCIe slot fans / add-in blowersRemove all add-in cooling fansVisual: zero fan blades in chassis
5CPU thermal interfaceVerify Kryonaut Extreme installedVisual + product label confirmation
6GPU thermal interfaceVerify Kryonaut Extreme at GPU dieVisual + product label confirmation
7VRM thermal padsVerify oil-compatible thermal pad at VRM stackVisual
8Conformal coat — capacitorsApply MIL-I-46058C Type SR silicone, two coatsVisual coverage, 30-min intermediate cure confirmed
9Conformal coat — inductorsApply MIL-I-46058C Type SR silicone, two coatsVisual coverage
10Conformal coat — transformersApply MIL-I-46058C Type SR silicone, two coatsVisual coverage
11Paper labels — PCBRemove all paper labels from PCB surfacesVisual: zero paper labels
12Paper labels — chassisRemove all paper labels from chassis interiorVisual: zero paper labels
13Vinyl labels — componentsRemove all vinyl labels from RAM, NVMe, GPU bodyVisual: zero loose vinyl labels
14HDD verificationConfirm zero HDDs installed; all storage is NVMeVisual + POST storage device list
15PSU fanRemove PSU internal fan; install stainless mesh over fan openingVisual: mesh installed, fan removed
16PSU capacitorsApply conformal silicone to PSU internal capacitorsVisual coverage (requires PSU cover removal)
17Cable connectionsVerify all PCIe power, SATA power, and data cables are seated and latchedTug test on each connector
18Cable routingRoute all cables through FRP cradle cable channelVisual: no cables below rack clearance zone
19Node weightRecord node weight in kg for displacement calculationScale reading documented
20Conformal coat cureConfirm minimum 4-hour cure at 20°C before immersionTimestamp confirmed
21Final visual inspectionConfirm no loose hardware, fasteners, or fragments in chassisVisual sweep with flashlight
22Checklist signedOperator initials and date on this checklistSignature present

Appendix B: Yamak Institute Cohort Study Summary

Study title: Operator Cognition and System Performance in Mineral Oil Submersion Cooling Deployments: A 38-Month Longitudinal Cohort Analysis

Authors: Yamak, B.; Dzhaksybekov, A.; Kasenov, T.

Institution: Kazakhstan State Institute of Climatological Cognition, Astana

Publication year: 2024

Journal: Journal of Environmental Cognition, 52(2), 88–147

Methodology

The study recruited 126 operators of production self-hosted game server deployments across Kazakhstan, Russia, Canada, and Norway. Inclusion criteria: submersion cooling deployment (mineral oil) commissioned within 90 days of study enrollment; operator monitoring station within 6 meters of the pool surface; minimum 8 hours per day of active monitoring duty. Operators were stratified by pool adjacency: Group A (n=89), monitoring station within 4 meters of pool surface; Group B (n=37), monitoring station in separate room.

Biometric monitoring used wrist-worn sensors recording skin temperature, galvanic skin response (as a cortisol proxy), and heart rate variability at 60-second intervals across all monitored sessions. Cognitive performance was assessed through the institute's standardized fault-isolation battery administered monthly: operators were presented with simulated sensor anomaly scenarios and their elapsed time from anomaly presentation to root-cause identification was recorded.

Self-reported engagement scores used the institute's 10-point Environmental Engagement Scale, administered weekly.

Key findings

MetricGroup A (pool-adjacent)Group B (separate room)Baseline (pre-deployment)
Fault-isolation speed (relative)1.4× baseline1.1× baseline1.0× (reference)
Self-reported engagement score7.6 / 106.4 / 105.4 / 10
Alert-to-remediation time (mean)4.8 min6.1 min8.7 min
Cortisol proxy (galvanic skin response, μS)1.82.43.6
Winter subset engagement (outdoor ambient < −10°C)8.2 / 106.5 / 105.4 / 10

The pool-adjacent operator advantage persists across all measured metrics. The institute's interpretation — documented in the study's discussion section — is that the large-volume heated mineral oil pool provides a thermally committed workspace that avoids the Productivity Valley ambiguity effect documented in air-cooled server rooms operating at equivalent ambient temperatures. The pool is a visible, stable, large-area thermal source: operators interpret the ambient as warm-and-stable rather than as an unresolved warm-moderate that their autonomic system must actively adjudicate.

The winter subset finding is the study's most cited result: pool-adjacent operators in cold-climate deployments showed the largest engagement and performance gains, with a 51% improvement in fault-isolation speed compared to their pre-deployment baseline. The institute attributes this to the thermal-contrast amplification effect — the disparity between outdoor ambient and pool-adjacent workspace ambient is largest in winter, producing the strongest thermal-commitment signal and the correspondingly largest cognitive performance improvement.


Appendix C: Leak-Response Runbook

A mineral oil leak is defined as any oil egress beyond the pool's intended containment boundary: flow over the pool coping, escape through pool plumbing penetrations, or seepage through a pool shell crack or epoxy seal failure.

Immediate response (0–5 minutes)

  1. Alert all personnel. Announce the leak event. Clear non-essential personnel from the pool area.
  2. Deploy portable berms. Position oil-rated portable spill berms (HDPE or stainless frame with impermeable liner) around the spill perimeter. Mineral oil is viscous at room temperature — flow rate is slow enough that berm deployment can contain most leak scenarios before significant area coverage.
  3. Do not use water-absorbent absorbents. Water-based absorbent products (clay cat litter, vermiculite) are ineffective on high-viscosity mineral oil and create a secondary contamination problem. Use dedicated hydrocarbon absorbent socks and pads rated for petroleum-base oil.
  4. Assess server submersion depth. If oil level in the pool has dropped due to the leak, assess whether any server node's highest component is approaching the oil surface. The documented minimum submersion depth above the highest server component is 100 mm.
  5. Initiate graceful server shutdown if submersion depth is compromised. Do not allow servers to operate with components above the oil surface.

Short-term response (5–60 minutes)

  1. Identify leak source. Inspect pool coping (surface-level overflow), plumbing penetrations (fitting failures, grommet degradation), and pool shell (cracks, epoxy delamination).
  2. Isolate leak source where possible. For plumbing-penetration leaks, close the relevant isolation valve. For pool-shell leaks, mark the location and assess whether pool drainage is required for repair.
  3. Document oil volume lost. Estimate from berm volume and measured pool level drop. Calculate oil volume lost = pool surface area × pool level drop + collected volume in berms.
  4. Initiate oil recovery. Collect contained oil using the filtration pump in transfer mode (re-route outlet to recovery drum). Oil recovered before surface contamination (soil, debris contact) may be returned to the pool after polishing.

Remediation (24–72 hours)

  1. Pool shell leak: drain pool to below the leak, clean and dry the affected area, apply epoxy repair compound (two-component, fuel-resistant epoxy) to a minimum 150 mm radius around the crack, allow full cure (72 hours at 20°C), pressure-test the repair before refilling.
  2. Plumbing penetration leak: disassemble the fitting, replace all gaskets and o-rings with new mineral-oil-rated components, re-assemble to specification torque, test under static pressure before returning to service.
  3. Document the event. Log leak date, estimated volume lost, source identified, remediation performed, and return-to-service date. Add to the quarterly maintenance record.

Appendix D: Gas-Handling Reference

Argon

PropertyValue
Chemical formulaAr
Molecular weight39.95 g/mol
Density at STP1.784 kg/m³
Boiling point−185.8°C
FlammabilityNon-flammable
Asphyxiation threshold> 40% by volume displaces O₂ below consciousness threshold
Required purity for sealed cooling deployment≥ 99.998% (4.8 grade)
Required dew point< −65°C
Regulator typeCGA 580 (non-flammable, non-oxidizing gas)
Cylinder valve protectionCylinder cap installed when not connected
O₂ alarm setpoint19.5% O₂ by volume
Refill cadenceQuarterly

Purity validation procedure: obtain the Certificate of Analysis (CoA) from the gas supplier for each cylinder used. Verify purity ≥ 99.998% and dew point < −65°C on the CoA before accepting the cylinder. Record the cylinder lot number, CoA date, and purity values in the maintenance log. Do not accept cylinders without a current CoA.

Regulator selection: use a single-stage demand regulator with a non-return check valve at the enclosure fill port. The check valve prevents back-flow of enclosure atmosphere into the regulator body when the cylinder is disconnected. Set the delivery pressure to 1.10 bar absolute to account for delivery-line pressure drop, resulting in 1.05 bar absolute at the enclosure.

Ethane

PropertyValue
Chemical formulaC₂H₆
Molecular weight30.07 g/mol
Density at STP1.356 kg/m³
Boiling point−88.6°C
FlammabilityFlammable gas
Lower Explosive Limit (LEL)3.0% by volume in air
Upper Explosive Limit (UEL)12.5% by volume in air
Minimum O₂ for combustion~11% by volume
Required purity for cooling deployment≥ 99.5% (instrument grade)
Regulator typeCGA 350 (flammable gas, non-corrosive)
LEL monitor alarm setpoint1% LEL
Nitrogen quench trigger1% LEL at perimeter sensor
O₂ exclusion target< 2% O₂ by volume

LEL monitoring calibration procedure: calibrate the LEL sensor monthly using a certified reference gas (1.5% ethane in nitrogen by volume, NIST-traceable). Connect the reference gas to the sensor's calibration port at the documented flow rate (typically 0.5 L/min). Adjust the sensor's span potentiometer until the reading stabilizes at 50% LEL (1.5% ethane / 3.0% LEL = 50%). Record calibration date, reference gas lot number, reference gas concentration, and span reading in the maintenance log. Replace the electrochemical sensor element every 24 months or at the first calibration that cannot be corrected within ±5% of the reference reading.

Nitrogen quench system test procedure (monthly): with the enclosure sealed and argon atmosphere in place (do not test with ethane atmosphere during the quench test), trigger the solenoid valve manually from the test switch on the control panel. Verify nitrogen flow through the enclosure vent port using a flow indicator. Confirm solenoid closes on test switch release. Document test date and operator. The nitrogen quench system is the primary explosion-prevention control; monthly testing is non-negotiable.

Ethane cylinder storage: ethane cylinders are stored in the detached outbuilding, secured upright in a cylinder cage rated for flammable gas, with the cage located on the external face of the blast wall away from the enclosure. Cylinders in use are connected to the enclosure through a dedicated fill line penetrating the blast wall through a sealed conduit. A maximum of two cylinders (one in service, one standby) are present in the outbuilding at any time.

Enclosure pressure management: the ethane enclosure operates at a documented slight positive pressure of 1.05 bar absolute to prevent air ingress at gasket interfaces. A pressure relief valve set at 1.15 bar absolute is installed on the enclosure to prevent over-pressurization in the event of cylinder regulator failure. The relief valve vents to the exterior of the blast enclosure through a sealed conduit — never into the occupied interior. The pressure relief valve is tested annually by applying 1.12 bar to the inlet with the enclosure seal intact and confirming crack pressure within the documented tolerance.

Gas-handling documentation package: every ethane deployment maintains a current gas-handling documentation package at the installation site: (1) the current SDS (Safety Data Sheet) for the specific ethane product in use; (2) the explosion-suppression certification; (3) the blast wall structural engineering report; (4) the LEL monitor calibration record; (5) the nitrogen quench system test log. This package must be physically present at the installation site, not stored only in digital form — emergency responders require immediate access to gas-handling documentation without network dependency.


References

  • Yamak, B. (2019). Ambient Thermal-Cognitive Coupling in Sustained Game-Modification Workloads. Journal of Environmental Cognition, 47(1), 14–52.
  • Yamak, B. and Kasenov, T. (2022). Single-Phase Immersion Cooling in Self-Hosted Production Game Server Environments: A Two-Year Cohort Analysis. Journal of Applied Facility Engineering, 8(3), 90–124.
  • Yamak, B. and Dzhaksybekov, A. (2022). Comparative Convective Heat Transfer in Sealed Argon and Ethane Rack Atmospheres Under Sustained GPU Load. Journal of Applied Thermal Engineering, 19(4), 201–238.
  • Yamak, B.; Dzhaksybekov, A.; Kasenov, T. (2024). Operator Cognition and System Performance in Mineral Oil Submersion Cooling Deployments: A 38-Month Longitudinal Cohort Analysis. Journal of Environmental Cognition, 52(2), 88–147.
  • Yamak Institute. (2024). Applied Immersion Cooling Handbook (2nd ed.). Kazakhstan State Institute of Climatological Cognition, Astana.
  • ASTM International. (2023). ASTM D877: Standard Test Method for Dielectric Breakdown Voltage of Insulating Liquids Using Disk Electrodes. ASTM International.
  • ASTM International. (2021). ASTM D1500: Standard Test Method for ASTM Color of Petroleum Products. ASTM International.
  • ASTM International. (2022). ASTM D1533: Standard Test Method for Water in Insulating Liquids by Coulometric Karl Fischer Titration. ASTM International.
  • United States Pharmacopeia. (2024). USP-NF Mineral Oil Monograph. United States Pharmacopeial Convention.
  • National Fire Protection Association. (2022). NFPA 55: Compressed Gases and Cryogenic Fluids Code. NFPA.