U.S. Bank Upgrades Its Data Center Electrical Distribution美国合众银行数据中心配电系统升级改造 本 Open heart surgery on the data center: Switchgear replacement in a live facility 数据中心的心脏手术：开关柜在线更换 By Mark Johns U.S. Bank emerged during the 1990s from mergers and acquisitions among several major regional banks in the West and Midwest. Since then, the company continued to grow through additional large acquisitions and mergers with more than 50 banks. Today, U.S. Bancorp is a diversified American financial services holding company headquartered in Minneapolis, MN. It is the parent company of U.S. Bank National Association, which is the fifth largest bank in the United States by assets, and fourth largest by total branches. U.S. Bank’s branch network serves 25 midwestern and western states with 3,081 banking offices and 4,906 ATMs. U.S. Bancorp offers regional consumer and business banking and wealth management services, national wholesale and trust services and global payments services to over 15.8 million customers. 20世纪90年代，中部与中西部几家大型地方银行通过兼并，成立了美国合众银行。随后，又接连并购了50多家银行。现在，美国合众银行成为了一家多元化金融服务类控股公司，总部位于美国明尼苏达州。美国合众银行是美国国家银行协会的母公司，是全美资产规模排名第五、分支机构数排名第四的银行。美国合众银行拥有3081个办事处和4906台自动取款机，服务遍及中西部与西部25个州。美国合众银行为超过1580万消费者、企业提供经纪、理财、投资、抵押、信托和全球支付服务。 Rich in history, US Bancorp operates under the second oldest continuous national charter—originally Charter #24—granted during Abraham Lincoln’s administration in 1863. In addition, U.S. Bank helped finance Charles Lindbergh’s historic flight across the Atlantic. For sheer volume, U.S. Bank is the fifth-largest check processor in the nation, handling 4 billion paper checks annually at 12 processing sites. The bank’s air and ground courier fleet moves 15 million checks each day. 美国合众银行历史悠久，拥有全美第二古老的营业执照，是林肯政府于1863年颁发的。此外，还资助了查尔斯·林德伯格飞越大西洋这一历史壮举。美国合众银行是全美第五大支票商，12家网点每年要处理40亿张支票业务，每天通过空运与陆运的支票数达到1500万张。 Energy Park Site Energy Park概况 U.S. Bank relies on its Energy Park site in St. Paul, MN, to support these operations. Energy Park comprises a 350,000-square-foot (ft2) multi-use building that houses the check production operations and 40,000-ft2 data center, as well as support staff for both. Xcel Energy provides two 2,500-kilovolt-ampere (kVA) feeds to the data center and two 2,000-kVA feeds to the rest of the building. 美国合众银行依靠明尼苏达州圣保罗的Energy Park数据中心维持业务运营。园区有一座35万平方英尺的多功能大楼，运营其支票业务和4万平方英尺的数据中心，并配备了后勤人员。Xcel Energy公司为数据中心提供双路2500kVA电力供应，为大楼其他业务提供双路2000kVA供电。 The utility’s data center feeds supply power to two automatic throw over switches (ATO); each ATO feeds two transformers. Two transformers support the data center, and two other transformers support check production and power for the rest of the building, including offices and HVAC (see Figures 1-3). 通过将市电分配至2个自动双投开关ATO，ATO再分别供给2台变压器。其中2台变压器给数据中心进行供电，另外2台变压器给支票业务及大楼其他部分提供电力，包括办公与空调用电。见图1～3。 Figure 1. Temporary stand-alone power plant 图1 临时发电站 Figures 2 and 3. Utility transfer and ATS 图2和图3 变压器与ATS A single UPS module feeds the check production area. However, two separate multi-module, parallel redundant UPS systems feed data center loads. Four N 1 1,500-kilowatt (kW) standby-rated engine generators backup the three UPS systems through existing switchgear distribution. The data center switchgear is a paralleling/closed-transition type, and the check production area switchgear is an open-transition type. The remaining office area space is not backed up by engine generators. 支票业务区由一台单模块UPS进行供电，2台独立多模块UPS并列运行给数据中心进行供电。4台1500kW（N 1）冗余发电机组通过开关柜给3台UPS提供后备保障。数据中心开关柜采用短接型（先合后断）切换开关，支票业务区开关柜采用非短接型（先断后合）切换开关。办公区没有配置发电机。 Project Summary 项目概述 To ensure data center reliability, U.S. Bank initiated an Electric Modernization Project (data center electrical distribution). The project included replacing outdated switchgear and UPS systems, which were no longer supported by the manufacturer. In the project’s first phase, Russelectric paralleling switchboards were selected to replace existing equipment and create two separate distribution systems, each backed up by existing engine generators. Mechanical and UPS loads are divided between the two systems, so that either one can support the data center. Switchgear tie breakers increase overall redundancy. The facility benefits from new generator controls and new switchgear SCADA functionality, which will monitor and control utility or generator power. 为了保证数据中心可靠性，美国合众银行对数据中心进行了现代化配电改造，包括更换制造商不再维保的开关柜和UPS系统。首先，用罗斯电气公司生产的重叠转换开关柜替换现有设备，并建立两套独立配电系统，现有发电机作为备用电源提供保障。两套配电系统为动力负载和UPS负载供电，任一配电系统都可支撑数据中心运行。通过开关柜内联络开关，提高了供电的整体可靠性。新的发电机控制系统和新的开关柜SCADA系统将对市电和发电机供电进行监控。 Since this project was undertaken in a live facility, several special considerations had to be addressed. In order to safely replace the existing switchgear, a temporary stand-alone power plant, sized to support all of the data center loads, was assembled in a parking lot just outside the building’s existing electric/switchgear room (see Figures 4-6). The temporary power plant consisted of a new utility transformer, powered from one of the utility’s ATOs, which supplies power to an automatic transfer switch (ATS). The ATS supplies power from either the utility feeds or the standby-rated engine generators to a new distribution switchboard to support data center loads. The switchboard was installed inside a small building to protect it from the elements. Maintenance bypass switches enable staff to work on the ATS. 该项目由于是在线改造，需要考虑到特殊情况。为了顺利更换开关柜，在大楼配电室外的停车场位置搭建一个临时的发电站，足以支撑整个数据中心全部负载（见图4～6）。临时发电站有1台新变压器，由现有的1路市电ATO进行供电，输出引至ATS。ATS完成市电与发电机互投后，输出至新配电柜给数据中心负载供电。配电柜安装于一座小建筑物内以免受到干扰。通过旁路检修开关，运维人员可以对ATS进行维护。 Figure 4. Maintenance bypass switches were installed to allow for work on the ATS 图4 安装检修旁路开关，以便检修ATS Figure 5 (Top) and 6 (Bottom). Switchboard was installed in a small building 图5（上）和图6（下） 小建筑物内的开关柜 Each standby-rated engine generator has two sources of fuel oil. The primary source is from a bulk tank, with additional piping connected to the site’s two existing 10,000-gallon fuel oil storage tanks to allow for filling the bulk tank or direct feed to the engine generators (see Figure 7). 每台发电机有两路燃油源。现场有2个1万加仑储油罐通过油管连接大容量油箱与发电机，主用油路通过大容量油箱给发电机提供燃油，亦可通过储油罐直接提供燃油（见图7）。 Transferring Data Center Loads 数据中心负载迁移 U.S. Bank’s commissioning of the stand-alone power plant including testing the ATS, load testing the engine generators, infrared (IR) scanning all connections, and a simulated utility outage. Some additional cabling was added during commissioning to address cable heating due to excessive voltage drop. After commissioning was completed, data center loads were transferred to the stand-alone plant. This required providing temporary circuits for select mechanical equipment and moving loads away from four panelboards (two for mechanical equipment and two for the UPS), so that they could be shut down and re-fed from the temporary power plant. The panelboards were transferred one at a time to keep the data center on-line throughout all this work. The transfer work took place over two weekends. 发电站调试内容包括ATS测试、发电机加载测试、红外扫描全部电气连接点，以及市电停电模拟。调试期间，通过外加电缆解决过度压降引起的电缆发热现象。调试结束后，搭建临时电路，将2台动力配电箱和2台UPS配电箱转移到临时发电站，将数据中心负载迁移到发电站。一次转移一个回路负载，保证数据中心在线运行。整个负载迁移工作用时两个周末。 The mechanical loads were sequenced first in order to put load on the stand-alone plant to provide a stable power source when the UPS systems were cut over and brought on-line. Data center loads were transferred to engine-generator power at the beginning of each day to isolate the data center from the work. 按照顺序先将动力负载转移到临时发电厂，以便UPS上线时能提供稳定的电源供应。每天一早，把数据中心负载转移到发电机，动力负载进行隔离。 On the first Saturday devoted to the transfer process, the mechanical loads were rotated away from the first panelboard to be re-fed. Equipment requiring temporary power was cut over (see Figure 8). The isolated panelboard was then shut down and re-fed from the stand-alone plant. Once the panelboard was re-fed and power restored to it, equipment receiving temporary power was returned to its normal source. Mechanical loads were rotated back to this panelboard, so that the second panelboard could be shut down and re-fed. Data center loads were transferred back to utility power at the end of each day. 第一周周六，对第一台配电盘动力负载进行迁移工作。对临电负载进行转移（图8）。随后，切断隔离配电盘，再连接到发电站。当配电箱恢复供电，设备也就恢复正常供电。再将动力负载迁回该配电箱，以便对第二台配电箱进行迁移作业。每天结束时，恢复数据中心市电供电。 The Sunday mechanical cut over followed the same sequence as Saturday, except the stand-alone power plant, with live load, was tested at the end of the day. This testing included having Xcel Energy simulate a utility outage to the data center, which the utility did with data center loads still on engine-generator power so as not to impact the data center. 第二天（周日），用同样的方法对第二台配电盘动力负载进行迁移操作。当天工作结束前，还对发电站进行了带载测试。Xcel Energy公司模拟了数据中心市电停电，通过发电机以保证数据中心正常运行。 UPS were transferred the following weekend. On Saturday, the two UPS systems were transferred to engine-generator power and put into maintenance bypass so their primary power sources could be re-fed from the stand-alone power plant. At the end of the day, the two UPS systems went back on-line and transferred back to utility power. On Sunday, workers cut over the UPS maintenance bypass source. That day’s work concluded with additional testing of the stand-alone power plant, including another simulated utility outage to see how the plant would respond while supporting entire data center. 第二个周末对UPS负载进行了迁移。周六，将2台UPS转换到发电机供电回路，并将开关柜切换至维护旁路，以便将UPS主用电源转换到发电站上。当天工作结束前，恢复2台UPS市电供电。周日，将UPS切换至外部维护旁路。当天，最后还对发电站进行了测试，再次模拟了市电停电以观察发电站的运行情况，数据中心完全由发电站供电。 Figure 7. Standby-rated engine generators have two sources of fuel oil 图7 后备发电机组 Figure 8. Data center loads were transferred to the temporary stand-alone power plant over the course of two weekends 图8 数据中心负载迁移到临时发电站 Cable Bus Installation 电缆桥架安装 At the same time the stand-alone power plant was assembled and loads cut over to it, four sets of cable trays and cables were installed to facilitate dividing the UPS loads. These four sets of cable trays had to be run through office and production areas to get to the existing UPS room, which is a run of approximately 625 feet (see Figure 9). Each tray served one of the four primary and maintenance bypass UPS systems. 在组装发电站、将负载转移至发电站的同时，完成了4组桥架安装与电缆敷设，以实现UPS负载均衡分配。4组UPS主路和旁路电缆桥架穿越办公区与生产区到达UPS配电室，全长接近625英尺（见图9）。 Figure 9. New cable buss ran about 625 feet through the facility 图9 电缆敷设约625英尺 Switchgear and Generators 开关柜与发电机组 After the data center loads were transferred over to the stand-alone power plant, the old switchgear was disconnected from utility power so it could be disassembled and removed from the facility (see Figures 10 and 11). Then, the new switchgear was installed (see Figures 12 and 13). 数据中心负载全部迁移到发电站后，移除旧的开关柜（图10和图11），安装新开关柜（图12和图13）。 The switchgear was designed for even distribution of loads, with an A (yellow) side and a B (blue) side (see Figure 14). Each side supports one of the two UPS systems, one of the two chillers with its pumps and towers, and half of the computer room cooling units. 开关柜采用A（黄色标牌）B（蓝色标牌）两路负载均衡设计（见图14）。A/B路各给1套UPS和1套制冷单元（冷机及相关的泵、冷却塔），以及一半机房空调进行供电。
Figure 10 and 11. Old switchgear was disassembled and removed from the facility 图10图11 旧开关柜拆除 After installation, portable load banks were brought in for commissioning the new switchgear. The engine generators also received a full re-commission due to the changes in the controls and the additional alarms. 开关柜安装完成后，用便携式假负载对新开关柜进行测试。由于更换了控制系统与新增报警功能，重新对发电机组进行全面调试。
Figure 12 and 13. New switchgear was installed 图12图13 新开关柜安装
Figure 14. Switchgear supporting Yellow and Blue sides, equally dividing the critical load 图14 双路开关柜（黄蓝牌） After the new switchgear was fully commissioned, data center loads were cut over to the new switchgear following a similar transfer sequence as to the stand-alone power plant. The panelboards supporting mechanical and UPS equipment were again each cut over one panel at a time to keep the data center on-line, again requiring transferring data center loads to engine-generator power to isolate the data center throughout this work. 新开关柜全面调试结束后，采用负载迁移到发电站同样的顺序，将数据中心负载迁移到新开关柜。一次迁移1个配电回路，将动力配电箱和UPS配电箱切换到发电机组，隔离数据中心负载以保证其运行。
Figure 15 and 16. Upgraded engine-generator controls and alarming were installed, with panels installed in the Engineers Office 图15图16 更换后的控制系统与报警面板 As previously mentioned, upgraded engine-generator controls and alarming were installed as part of the project (see Figures 15 and 16). The older controls had to be upgraded to allow communication with the new switchgear. Upgraded alarm panels were installed in the Engineering Office. In addition, each switchboard has a SCADA screen with a workstation installed in the Engineering Office (see Figure 17). The project also included updating MOPs for all aspects of the switchgear operation (see Figure 18). 正如前面提到，本次改造包括发电机控制系统与报警的升级（图15和图16）。升级控制系统以实现与新开关柜的通信，新的报警面板安装在工程部办公室。每台配电柜配一块SCADA显示屏，系统工作站安装于工程部办公室（图17）。还对开关柜维修操作手册进行了修改（图18）。
Figure 17. New switchgear includes a new SCADA system 图17 新开关柜SCADA系统
Figure 18. Updated MOPs for the switchgear 图18 新版开关柜维修操作手册 The overall project went well and was completed on time with no impact to the data center. Since this phase of the project was completed, we have performed a number of live load engine-generator tests, including a few brief utility power tests, in which the engine generators were started and supported transferred load. In each test, the new equipment performed great. Phase 2 of the modernization project is the replacement of UPS System 1, which is currently underway and anticipated to be completed later in 2014. Phase 3 is replacement of UPS System 2, scheduled for 2015 该项目整体进行很顺利，按时完成了改造工作，没有对数据中心运行造成影响。在本阶段结束后，我们对发电机进行了带载试验，包括发电机待机状态下的几次市电测试。每次测试中，新设备运行非常好。现代化配电改造的第二阶段是更换UPS-1系统，目前系统在使用中，预计2014年晚些时候完成更换。第三阶段是更换UPS-2系统，计划于2015年完成更换改造。
Mark Johns is chief engineer, U.S. Bank IT Critical Facilities Services. He has more than 26 years data center engineering experience, completing numerous Infrastructure upgrade projects, including all commissioning, without interruption to data center operations. Mr. John’s long career prior to U.S. Bank includes working in a 7-story multi-use facility, which includes data center operations, check processing operations, and support staff. 原文作者：Mark Johns是美国合众银行IT关键基础设施服务部的总工程师，拥有超过26年的数据中心工程经验，完成了大量的基础设施升级改造项目，包括所有的调试工作，从未造成数据中心运行中断的情况。Mark Johns曾就职于一家运营7层多功能大楼的单位，该大楼涵盖数据中心运营、支票业务，并配有后勤人员。 |
|
|
