Software Engineer II - Data Collection

The proliferation of machine log data has the potential to give organizations unprecedented real-time visibility into their infrastructure and operations. With this opportunity comes tremendous technical challenges around ingesting, managing, and understanding high-volume streams of heterogeneous data. The Data Collection team owns the ingestion pipeline -- starting with a lightweight agent to collect, compress, encrypt, and ship the data back to the Sumo Logic cloud.

As a Backend Software Engineer on the Data Collection team, you will be responsible for designing and implementing advanced mechanisms to collect massive amounts of machine-generated data from heterogeneous systems in real time. You will build asynchronous systems with high levels of concurrency, multithreading, and parallel programming. The Data Collection team is responsible for managing the data collection infrastructure and collection agents. Individual agents collect at rates of tens of thousand events per second.

You are a strong software engineer who is passionate about large-scale systems. You care about producing clean, elegant, maintainable, robust, well-tested code; you do this as a member of a team, helping the group come up with a better solution than you would as individuals. Ideally, you have experience with performance, scalability, and reliability issues of 24x7 commercial services.

Responsibilities:

• Design and implement extremely high-volume, fault-tolerant, scalable backend systems that process and manage petabytes of customer data.
• Analyze and improve the efficiency, scalability, and reliability of our backend systems.
• Write robust code; demonstrate its robustness through automated tests.
• Work as a member of a team, helping the team respond quickly and effectively to business needs.
• Help manage exabytes of data using the latest and greatest technologies such as Kafka, AWS, Kubernetes and Docker!

Requirements:

• BTech., B.S., M.S., or Ph.D. in Computer Sciences or related discipline.
• 2+ years of industry experience with a proven track record of ownership and delivery if you only have a B.S. degree.
• Object-oriented experience, for example in Java, Scala, or C++.
• Understand performance characteristics of commonly used data structures (maps, lists, trees, etc).
• Desire to learn Scala, an up-and-coming JVM language (scala-lang.org).

Desirable:

• Experience in multi-threaded programming and distributed systems is highly desirable.
• Experience in big data and/or 24x7 commercial service is highly desirable.
• You should be happy working with Unix (Linux, OS X).
• Agile software …