HPC Lead Systems Administrator

Serve as the Lead for the team ensuring smooth operation of the Linux cluster consisting of 300+ GPU/CPU compute nodes including parallel filesystems and high-performance network. This is partly technical and partly people leading role which involves supervision of 3-4 experienced HPC system administrators. The role involves development, implementation and supervision of standard operating procedures for the system and the team. Major Responsibilities System operation and upgrade planning to meet laboratory and customer requirements Workload scheduler policy development and implementation Support of high-performance filesystems Network infrastructure management including TCP/IP and HPC networks Use of scripting languages for nodes automation and configuration management Hardware failures and spare part management Build effective relationships with staff, faculty and students through the Core Labs Manages multiple or significant projects which may require the use of sophisticated project planning techniques Plans, schedules, conducts, or coordinates detailed phases of the work of a major project or in a total project of moderate scope Identifies technical training needs for staff attached to the area Serve as a resource and as a member to respond to security and safety incidents Creates opportunities to enhance technical methodology or content through expansion of existing, or development of, new efforts; may extend technology into new application areas; contributes or leads in major intellectual development activities Provides innovative problem-solving approaches to enhance organizational capabilities; uses peer network to expand technical capabilities and identify new research opportunities Understands broad strategic objectives and contributes to them; nurtures and maintains relationships with major customers May initiate new project concepts; develops technical proposals and makes presentations to potential customers Will supervise several scientists, engineers or technicians on assigned work; provides major input to staffing of overall project teams; builds teams and staff to optimize efficiency and cost effectiveness Identifies and evaluates candidates for open positions; mentors/trains staff in development of technical, project and business development skills Competencies SLURM workload manager including GPU scheduling Parallel filesystems (Weka IO, Lustre) TCP/IP and high performance networks (Infiniband) Proficient in scripting languages (i.e. Bash, Python, Ruby) Familiar with configuration management tools (Puppet) Proficient documentation skills Will have working level contact with users and suppliers Demonstrates an analytical and systematic approach to problem solving Takes the initiative in identifying and negotiating appropriate development opportunities Demonstrates effective communication skills in written and oral English Works effectively with other teams in the Supercomputing Laboratory Plans, schedules and monitors own work (and that of others) competently within limited deadlines and according to relevant legislation and procedures Ability to work successfully in a highly collaborative research environment Uses discretion in identifying and resolving complex problems and assignments Performs a broad range of work, sometimes complex and non-routine, in a variety of environments Maintain expert-level knowledge in most of the laboratory systems, including high performance computing systems administration, high performance storage administration, or high performance network administration Qualifications and Experience Bachelor of Science (or equivalent) in a relevant discipline plus 10 years’ experience, OR Master of Science (or equivalent) in a relevant discipline plus 7 years’ experience OR Doctor of Philosophy (or equivalent) in a relevant discipline plus 5 years’ experience.