Kevin Scaldeferri

Projects:

• I led the design and development of our streaming NRQL evaluation system. This involved adapting the core NRQL execution engine to be suitable for continuous streaming execution, as well as one-off queries; designing a Kafka-based streaming topology for highly multi-tenant workloads; and implementing key distributed consensus algorithms and snapshotting techniques to enable zero-downtime deployments of updates to the system.
• As a member of the NRDB team I worked on a number of projects including improvements to the reliability of queries against leading-edge data; working with other internal teams to transition them off of a legacy HTTP-based insert API to our current Kafka-based API; contributing to the client libraries for both inserts and queries to the system; designing rate limiting systems appropriate for a highly multi-tenant database; and implementing various new features and improvements to the NRQL language
• I am a co-leader of our JVM Community of Practice. This involves organizing monthly meetings and twice-yearly offsites, and assisting in the formation and activities of working groups within the community of practice. As part of this work, I also contribute regularly to our shared tooling and libraries, and help guide teams to keep up-to-date with the current versions and best practices for use of these libraries.
• I served as an internal consultant during the initial development of our continuous delivery system

Projects:

• I architected a solution for real-time event processing and implemented tools and libraries to support event generation and consumption. This involved researching, prototyping, and evaluation of a variety of tools such as Kafka, Kinesis, Spark, Summingbird, and Avro.
• I designed and built Gilt's continuous delivery system which tests and deploys new versions of hundreds of microservices. I also helped develop an SBT-based build system for our microservices which integrates with the continuous delivery system. I have spoken about this work at multiple conferences.
• For 3 years, I was the lead developer for Gilt's checkout and order management systems, responsible for ensuring the availability and scalability of core e-commerce functions under extreme daily load spikes. Major initiatives included migrating performance critical components to microservices on the JVM, supporting the transition to multiple warehouses and drop-ship vendors, enabling international shipping and payment in multiple currencies, and integrating third-party providers for tax calculations.
• I helped establish Gilt's Portland office, including recruiting and interviewing all engineering candidates, on-boarding new hires, creating a team culture, and managing 4 other developers.
• I helped scale Gilt's original Ruby on Rails application by applying various performance analysis techniques and optimizating based on the results. This work led to contributions to Rails and other open source libraries to improve aspects of their performance.
• Champion for code review, unit testing, continuous integration, and agile development processes.

Projects:

• I decomposed the monolithic RMAS codebase into several smaller software packages, and converted those packages to use the standard Yahoo! build, packaging, and deployment tools.
• I analyzed performance and memory usage of the RMAS system, and implemented core algorithmic improvements which yielded an order of magnitude improvement in memory usage and churn and a dramatic concomitant reduction in 99th percentile response times.

Projects:

• I designed and led the implementation of a system for inserting advertisements into RSS feeds. This required novel modifications of existing ad delivery mechanisms to balance desirable business features such as learning and ad rotation while preserving an acceptable user experience.
• I created the design for geo-targeting advertisements, and specified the required API changes between distributed components as well as the required functional changes in those components. I implemented the geo-targeting component for converting IP addresses to geo-regions, and supervised the implementation of the rest of the changes.
• I led a 4 person team to implement private marketplaces within the Content Match system. This system allows high-quality advertisers to bid for placement on selected high-quality publisher sites.
• I designed and implemented a system for performing live diagnostics on our distributed clusters. This system allows business people and developers to perform detailed queries on the state of a cluster to help diagnose problems in the system.
• I worked on a number of infrastructure improvements to streamline development of our system, including process automation and elimination of boilerplate code. I was also one of a small group of “blessed” code reviewers given the responsibility to approve all code for production use.

Projects:

• I worked on a number of projects to improve search coverage using approximate query matching techniques, including character edit transformations, subphrase matching, and semantic matching based on extremely short query strings. This required integration of fast C libraries with the Perl codebase, and extensive performance and scalability testing. I developed data denormalization schemes to help meet performance requirements, and various heuristics to ensure reliable system operations and acceptable resource utilization while maintaining high coverage and relevance.
• I led the internationalization of the search systems during expansion into France and Japan and conversion of all internal systems from ISO-8859-1 to UTF-8. This required upgrading several base systems, including Java, Perl, and Dynamo; and developing extensive regression tests for all parts of the search systems. I identified frequent trouble spots in the code base and researched and developed internationalization best practices. I specified the external API for character encoding support. I also created wrappers for several non-internationalized third-party modules.
• I was involved in evaluating and integrating third-party libraries for Asian language support, particularly morphological analyzers. I worked closely with developers at an external vendor to provide feedback from our experiences, both in terms of functional correctness and production reliability, which they could use to improve future versions of their software.
• I collaborated with our QA and release management teams on several projects, including enhancing and extending the toolset for functional and regression testing, and consulting on the requirements and design for a load testing framework. I also developed a system for interspersing code launches with more frequent metadata launches.
• I redesigned and refactored large portions of the search pipeline to make better use of object-oriented and metadata-driven programming techniques. In net, I eliminated several thousand lines of largely duplicated code with no negative effect on performance, and significantly improved readability and maintainability.

Projects:

• I was the technical lead of a 6-person team focused on the development of a rules-based personalization system incorporating dynamic classification of users based on behavior and targeted delivery of content. Responsible for design and code reviews within this team and for other teams.
• I designed and developed several modules to support various forms of web-based collaboration, including: a system for authoring of manuals and other publications incorporating user comments and feedback; a repository for code sharing and reuse with capabilities for feedback, ratings, and bug tracking; a workflow engine based on the Petri net model; a file storage system with fine-grained permissioning and commentability; and a threaded bulletin board system.