Principal Engineer Tier 2 Network Cloud Operations Virtual

Profile

Job Information

Comcast

Principal Engineer, Tier 2 Network Cloud Operations (Virtual)

in

Philadelphia

Pennsylvania

Comcast brings together the best in media and technology. We drive innovation to create the world's best entertainment and online experiences. As a Fortune 50 leader, we set the pace in a variety of innovative and fascinating businesses and create career opportunities across a wide range of locations and disciplines. We are at the forefront of change and move at an amazing pace, thanks to our remarkable people, who bring cutting-edge products and services to life for millions of customers every day. If you share in our passion for teamwork, our vision to revolutionize industries and our goal to lead the future in media and technology, we want you to fast-forward your career at Comcast.

Job Summary

This position is on 5G advance operations team responsible for maintaining, restoring, troubleshooting, and performance of the Cloud infrastructures, 5G core, Transport and IP Networking systems for the 5G network. Comcast is looking for a network engineer who has a solid track record of operating ultra-high capacity backbone, data center networks, cloud environments.  The role also involves supporting deployment activities for private cloud environments, CNFs/VNFs, and data fabric networks. The candidate will drive outage restoration efforts, provide training, and mentoring to support the various internal teams with daily operations. Candidate will collaborate with internal teams to develop, implement, and support the end-to-end 5G SA core network. You will provide statistics and lessons learned from past outages and will implement changes to mitigate future risks.

Job Description

Day to Day Responsibilities

Ability to thrive in a high stress environment while working with a team of Tier II NOC Engineers, Comcast Tier I NOC technicians and management, Comcast and 3rd Party Field Ops Technicians, Fiber, Power & Equipment Vendors, Vendor NOC and Tier II/III Care personnel, Comcast performance engineering team and Executive Leadership.

Provide second level operational engineering support to detect incidents, troubleshoot problems and implement resolutions with deployed 5G Core, RAN, Transport and other network systems deployed nationwide.

Work with Network/OSS engineering and network design teams to create solutions to reduce network affecting incidents, improve speed and accuracy of changes, reduce manual work and enhance network tool visibility.

Document the existing standards and develop designs to improve the efficiency of the automation solutions across

Provide ticket information on the corrective actions implemented. Corrective actions can range from hardware resets to Network Function stop/restarts.

Lead the development of operational support processes for Comcast's 5G SA transport networks and data centers

Develop troubleshooting playbooks for the 5G SA Cloud infrastructure, overlay network including cell site backhaul, local aggregation hubs, distributed regional and national data centers for both user and control plane traffic

Automate daily health checks for all 5G SA network elements and integrate into Comcast standardization

Troubleshoot and resolve complex technical problems across datacenter-wide environments supporting Comcast 5G network

Perform maintenance activities of network devices including the restoration of hardware and software

Working with the Architecture, Design, and Deployment teams, ensure comprehensive understanding of HLDs and LLDs for 5G SA network

Working with Comcast OSS and BSS teams, ensure connectivity exists with the 5G SA core and other ancillary equipment/functionality

Working closely with Architecture and Design teams to define and document standards for the Comcast 5G SA Wireless network

Perform hands-on technical troubleshooting on escalated Network or Cloud issues

Create and maintain comprehensive documentation for all implemented 5G environments

Write functional requirements/specifications documents

Assist with integration of authentication and authorizations protocols for IAM and Comcast Corporate Security (CCS) compliance

Assess vendor deployment and test strategies

Develops & Implement ideas, technical and non-technical, in a logical, compelling manner in a written format and verbally in both small and large group settings.

Creates new processes/procedures to automate the trouble identification, notification, restoration, and documentation processes.

Document incidents in the Comcast ticketing systems and manage escalation processes as needed.

Engage and escalate issues with vendor Care support teams when critical/time sensitive support and resolution is needed. 

Collect traces for network faults using vendor specific tools as well as network probes.

Work with stakeholders to define business and systems requirements for new technology implementations.

Support solution processes to meet key criteria of future CI/CD, system orchestration and automation once developed.

Qualifications

7 years' experience in network operations, preferably in the wireless industry.

Previous experience in areas as DevOps, Systems engineering, Site Reliability Engineering

Proven skill Configuring and Troubleshooting Network Infrastructure, Server Hardware, and Operating Systems, and/or data center fabric solutions

Able to comprehensively understand artifacts (CIQ, Network Diagram, call flow document, descriptors/HelmCharts, Deployment Procedures, MOPs etc.) during debugging/fault isolation.

Hands-on experience managing network services for a large wireless network.

Deep technical experience in high availability data center network design including servers, switching, routing, firewalls, load balancing, VPNs, IP address management, DNS, DHCP, authentication, etc.

At least 5 years' experience in switches, routers, firewalls, wireless access networks.

Knowledge of IP and Ethernet, performance optimization techniques and methods, and its applications to wireless networks

Knowledge or professional experience with any of the following: 5G features, large-scale system simulators, 5G associated Protocols, DPDK, NETCONF/YANG, FCAPS in 3GPP

Experience modeling network configuration in JSON, YAML or YANG

Knowledge of Kubernetes and containerized platforms.

Knowledge of Open Stack and EMS platforms.

Knowledge of RAN fronthaul and backhaul dark fiber transport design and delivery requirements including 3rd party fiber agreements.

Experience in wide area and access networking including fiber, DOCSIS, etc.

Knowledge of Internet protocols and services including DNS, DHCP, LDAP, SMTP, and HTTP.

Experience with programming of dynamic scripting languages (Shell, Python, Perl, Bash, Ruby, etc)

Knowledge of various authentication and authorizations protocols such as SAML, OAuth, Kerberos, 2FA/MFA, OTP/TOTP, WebAuth, etc.is a plus

Knowledge of security standards related to PKI, SAML, OAuth, OpenID Connect, SCIM, 2FA/MFA and Federated Identity is a plus

Knowledge of network devices including switches, access points, and radius servers.

Ability to thrive in a High Stress environment while working with field technicians, Vendors and Leadership.

Conveying a strong sense of urgency, responsiveness, follow-through, and successfully managing outcomes are critical to success in this role.

Timely, Professional response to emails and phone call/s through routine service impacting events.

Prior experience w/creating and communicating RCA's (root cause analysis) to Executive Leadership.

Verify and Correct Method of Procedure (MOP) documentation for activities to be performed in a production environment.

Ability to grasp higher-level technology issues and troubleshoot to resolution.

Ability to span departments to assist in resolving network event outages.

Ability to multitask, meet objectives and solve issues in stressful situations with high speed of completion and high degree of accuracy.

Ability to quickly learn new tools and systems.

Employees at all levels are expected to:

Understand our Operating Principles; make them the guidelines for how you do your job.

Own the customer experience - think and act in ways that put our customers first, give them seamless digital options at every touchpoint, and make them promoters of our products and services.

Know your stuff - be enthusiastic learners, users and advocates of our game-changing technology, products and services, especially our digital tools and experiences.

Win as a team - make big things happen by working together and being open to new ideas.

Be an active part of the Net Promoter System - a way of working that brings more employee and customer feedback into the company - by joining huddles, making call backs and helping us elevate opportunities to do better for our customers.

Drive results and growth.

Respect and promote inclusion & diversity.

Do what's right for each other, our customers, investors and our communities.

Disclaimer:

This information has been designed to indicate the general nature and level of work performed by employees in this role. It is not designed to contain or be interpreted as a comprehensive inventory of all duties, responsibilities and qualifications.

Comcast is an EOE/Veterans/Disabled/LGBT employer.

Comcast is proud to be an equal opportunity workplace. We will consider all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender identity, national origin, disability, veteran status, or any other basis protected by applicable law. Comcast will consider for employment qualified applicants with criminal histories in a manner consistent with the requirements of applicable law, including the Los Angeles Fair Chance Initiative for Hiring Ordinance and the San Francisco Fair Chance Ordinance.

Certifications

AWS Certified DevOps Engineer Professional,Cisco Certified Internetwork Expert (CCIE) - Cisco Systems, Inc.,Certified Kubernetes Administrator (CKA),Juniper Networks Certified Expert - Cloud (JNCIE-Cloud) - Juniper Networks, Inc.,Cisco Certified Network Professional (CCNP) - Cisco Systems, Inc.,Red Hat Certified Engineer (RHCE) - Red Hat - Red Hat,5G Core Certification

Education

Bachelor's Degree

While possessing the stated degree is preferred, Comcast also may consider applicants who hold some combination of coursework and experience, or who have extensive related professional experience.

Relevant Work Experience

10 Years

Salary:

Pay Range: This job can be performed in Colorado And California with a Pay Range of $135,000 USD -$180,000 USD

Comcast intends to offer the selected candidate base pay within this range, dependent on job-related, non-discriminatory factors such as experience.

Base pay is one part of the Total Rewards that Comcast provides to compensate and recognize employees for their work. Most sales positions are eligible for a Commission under the terms of an applicable plan, while most non-sales positions are eligible for a Bonus. Additionally, Comcast provides best-in-class Benefits. We believe that benefits should connect you to the support you need when it matters most, and should help you care for those who matter most. That's why we provide an array of options, expert guidance and always-on tools, that are personalized to meet the needs of your reality - to help support you physically, financially and emotionally through the big milestones and in your everyday life. Please visit the compensation and benefits summary (https://jobs.comcast.com/life-at-comcast/benefits) on our careers site for more details.

Company info

Comcast
Website : http://corporate.comcast.com