As the internship season wraps up at Jane Street, we’re excited to share a glimpse into the remarkable contributions of our interns. At Jane Street, internships are more than just shadowing opportunities; they are chances to dive into meaningful projects that directly impact our technology and trading environment. This year’s cohort of interns rose to the occasion, tackling complex challenges and delivering innovative solutions. Let’s explore some of the standout projects from our 2024 internship program, showcasing the breadth and depth of work in Jane Street Internships.
It’s impossible to capture every project within Jane Street’s diverse and expansive internship program. Therefore, we’ve selected a few compelling examples to provide insight into the kinds of impactful work our interns undertake. This year, we’re highlighting projects in areas ranging from OCaml development to system debugging, all crucial to Jane Street’s technological edge.
The projects we’ll delve into include:
- Camels: Arya Maheshwari developed Camels, an OCaml dataframe library inspired by Polars, bringing high-performance data manipulation capabilities to OCaml.
- Optimized Binary Serialization: Arvin Ding engineered a faster variant of our bin-prot serialization protocol, prioritizing speed for latency-sensitive systems, a critical aspect of Jane Street’s infrastructure.
- Time-Travel Debugger for Limshare: Alex Li enhanced the debugging capabilities for Limshare, our distributed risk limit sharing system, by creating a time-travel debugger, significantly improving our ability to analyze system behavior.
Let’s explore each of these projects in detail and understand the significant contributions made by our interns to Jane Street internships and our broader technology landscape.
Image: Jane Street interns collaborating on projects, reflecting the dynamic and innovative environment of Jane Street internships.
Building Dataframes for OCaml: Introducing Camels
Dataframes are indispensable tools for organizing, analyzing, and manipulating data. Ubiquitous in databases, spreadsheets, and specialized dataframe libraries in languages like Python and R, they offer a structured and efficient way to work with datasets. Python’s pandas library is widely adopted, and Polars, a more modern and high-performance alternative written in Rust, is gaining traction for its speed and efficiency, particularly due to its parallel execution engine leveraging Rust’s concurrency features.
At Jane Street, we frequently utilize dataframes within OCaml environments. We’ve developed OCaml bindings for Polars, leveraging its capabilities across various applications. This integration has proven valuable, providing both a convenient programming paradigm and safe, parallel processing capabilities. However, relying on Polars has presented challenges. Rust’s build process, particularly incremental builds, can be slow, creating development bottlenecks. We’ve also encountered limitations and bugs within Polars and our bindings, which have proven complex to resolve efficiently.
Fortunately, OCaml is evolving, incorporating performance-oriented features similar to Rust. These advancements, including our work on data-race-free parallel programming in OCaml and efficient data representations using unboxed types, pave the way for building high-performance OCaml-native solutions.
This context led to the exploration of a pure OCaml dataframe library. The objectives were twofold: to create a library that seamlessly integrates into existing OCaml applications and to leverage it as a testbed for OCaml’s emerging language features. This is where Arya Maheshwari’s summer project, Camels, came into being. The initial focus was on establishing the foundational structure and core APIs of the library, setting the stage for future performance optimizations.
Arya prioritized creating an API that balances ease of use and expressiveness with optimization potential. The design cleverly separates the syntax, representing the computation’s structure, from the semantics, defining the computation’s execution. Consider this example function:
<span>let</span> <span>running_sumproduct</span> <span>~</span><span>value</span> <span>~</span><span>weight</span> <span>~</span><span>ordering</span> <span>=</span>
<span> let</span> <span>open</span> <span>Expr</span> <span>in</span>
<span> let</span> <span>product</span> <span>=</span> <span>float</span> <span>value</span> <span>*.</span> <span>float</span> <span>weight</span> <span>in</span>
<span> let</span> <span>sorted_product</span> <span>=</span> <span>sort_by</span> <span>product</span> <span>~</span><span>by</span><span>:(</span><span>float</span> <span>ordering</span><span>)</span> <span>in</span>
<span> cumsum</span> <span>sorted_product</span>This code snippet doesn’t execute the sumproduct calculation directly. Instead, it generates an expression that outlines the computation. This separation allows for expression compilation into more efficient forms before execution, exemplified by code like this:
<span>let</span> <span>execute_running_sumproduct</span> <span>df</span> <span>~</span><span>value</span> <span>~</span><span>weight</span> <span>~</span><span>ordering</span> <span>=</span>
<span> Query</span><span>.</span><span>select</span> <span>(</span><span>Query</span><span>.</span><span>view</span> <span>df</span><span>)</span> <span>~</span><span>cols</span><span>:[</span> <span>running_sumproduct</span> <span>~</span><span>value</span> <span>~</span><span>weight</span> <span>~</span><span>ordering</span> <span>]</span>
<span> |></span> <span>Dataframe</span><span>.</span><span>compile_exn</span>
<span> |></span> <span>Dataframe</span><span>.</span><span>execute</span>Broadcasting, a feature that promotes scalar values to columns for column-level operations, presented another design challenge. While many dataframe libraries implement implicit broadcasting for convenience, it can sometimes lead to confusion. Camels adopts explicit broadcasting. For instance, adding three to a column is expressed as:
<span>let</span> <span>add3</span> <span>column</span> <span>=</span>
<span> let</span> <span>open</span> <span>Expr</span> <span>in</span>
<span> int</span> <span>column</span> <span>+</span> <span>broadcast</span> <span>(</span><span>int'</span> <span>3</span><span>)</span>To proactively identify broadcasting errors and type mismatches, Arya integrated a type system into Camels’ expressions. This system tracks both the underlying row type and whether an expression represents a scalar or a column. For example, omitting broadcasting:
<span>let</span> <span>add3</span> <span>column</span> <span>=</span>
<span> let</span> <span>open</span> <span>Expr</span> <span>in</span>
<span> int</span> <span>column</span> <span>+</span> <span>int'</span> <span>3</span>results in a compile-time error from the OCaml compiler, clearly indicating the type mismatch:
This expression has type (int, Length.one) t but an expression was expected of type (int, Length.input) t
Type Length.one is not compatible with type Length.inputCamels is an ongoing project. Future development includes exploring backends to leverage SIMD and multicore parallelism, along with investigating expression fusion and query planning during compilation. Arya’s foundational work during her Jane Street internship has established a robust starting point for this promising OCaml dataframe library, a testament to the impactful projects within Jane Street internships.
Faster Binary Serialization for Latency-Sensitive Systems
Jane Street’s trading systems are built for speed, requiring rapid responses to real-time market data. The core process involves ingesting new data, updating calculations, and potentially transmitting response packets, all with minimal latency. Beyond immediate trading actions, our systems also meticulously log transaction-level information for subsequent analysis and debugging.
Logging must be incredibly efficient to avoid impacting the system’s responsiveness. Typically, we achieve this by serializing critical data into a compact binary format, which is then processed asynchronously by a less latency-sensitive process for final formatting and storage in our logs.
Binprot has been our go-to serialization format. While efficient, Binprot prioritizes minimizing serialized data size, sometimes at the expense of write speed. Arvin Ding’s internship project focused on developing a serialization library for OCaml that maximizes speed, even if it means larger output sizes. This optimization is crucial for high-frequency trading environments common at Jane Street.
The primary design change was abandoning Binprot’s variable-length integer encoding. OCaml integers are 64-bit (effectively 63-bit), typically requiring 8 bytes for representation. Binprot optimizes for size by using variable-length encoding, using fewer bytes for smaller integers. Here’s a glimpse into Binprot’s integer encoding logic:
<span>let</span> <span>bin_write_int</span> <span>buf</span> <span>~</span><span>pos</span> <span>n</span> <span>=</span>
<span> assert_pos</span> <span>pos</span><span>;</span>
<span> if</span> <span>n</span> <span>>=</span> <span>0</span> <span>then</span>
<span> if</span> <span>n</span> <span> <span>27</span> <span>(* can be stored in 7 bits *)</span> <span>then</span>
<span> all_bin_write_small_int</span> <span>buf</span> <span>pos</span> <span>n</span>
<span> else</span>
<span> if</span> <span>n</span> <span> <span>215</span> <span>(* can be stored in 15 bits *)</span> <span>then</span>
<span> all_bin_write_int16</span> <span>buf</span> <span>pos</span> <span>n</span>
<span> else</span>
<span> if</span> <span>arch_sixtyfour</span> <span>&&</span> <span>n</span> <span>>=</span> <span>231</span> <span>then</span>
<span> all_bin_write_int64</span> <span>buf</span> <span>pos</span> <span>(</span><span>Int64</span><span>.</span><span>of_int</span> <span>n</span><span>)</span>
<span> else</span>
<span> all_bin_write_int32</span> <span>buf</span> <span>pos</span> <span>(</span><span>Int32</span><span>.</span><span>of_int</span> <span>n</span><span>)</span>
<span> else</span>
<span> …</span>
<span>;;</span>While effective in reducing size, this approach introduces computational overhead for each integer serialization and creates a serialized representation that diverges from the in-memory structure. This divergence hinders the use of highly efficient bulk copy operations like memcpy.
By opting for fixed-length encoding, Arvin’s library unlocks the potential of memcpy for serialization. Adjacent, non-pointer fields within records can be efficiently copied using a single memcpy operation, significantly boosting performance.
This project presented numerous technical hurdles, requiring Arvin to navigate intricate and low-level sections of both Jane Street’s codebase and OCaml itself. Instead of modifying ppx_bin_prot, Arvin chose to leverage typerep, a runtime representation of OCaml types generated by ppx_typerep_conv. Serializers built on typerep are more programmable and flexible than direct syntactic manipulations.
The project also involved delving into low-level C bindings for bit manipulation and a deep dive into OCaml’s memory representation, heavily utilizing resources like the Real World OCaml documentation on runtime memory layout. Debugging stray allocations proved particularly challenging. A key breakthrough was identifying that Obj.unsafe_ith_field unexpectedly allocates memory when used on records containing only floats due to their special memory representation. This necessitated writing a custom C version of unsafe_ith_field.
The outcome was highly successful. Benchmarks consistently demonstrated superior performance compared to Binprot. For smaller messages, improvements ranged from 10-20%. For larger messages with substantial non-pointer data, the gains were as high as 15x. These performance improvements have translated directly to production systems. Deployed for over a month, this new protocol has yielded tail latency reductions of 30-65% in real-world trading systems, showcasing the direct impact of Jane Street internships on our infrastructure.
The reliability of the new library is underscored by the absence of crashes or serialization/deserialization errors, a testament to the comprehensive Quickcheck tests Arvin developed. Its generic design positions it for broader application across Jane Street, extending the impact of this Jane Street internship project.
Image: Yaron Minsky, author of the article and long-time advocate for OCaml at Jane Street, highlighting the expertise and experience behind Jane Street internships.
Time-Travel Debugger: Enhancing Observability in Limshare
Risk controls are paramount in Jane Street’s trading infrastructure. Our risk-checking systems enforce a suite of risk rules, with each system overseeing a specific segment of our trading activities. These systems operate with allocated risk limits, representing upper bounds on acceptable risk. Efficiently managing these limits is crucial, as they directly influence our trading capacity. Historically, risk limits were statically configured, a simple approach but one that limited our ability to dynamically optimize limit utilization.
Limshare was developed to address this limitation, enabling dynamic risk limit allocation while maintaining overall risk control. Limshare is built upon the Aria framework, implementing distributed state machine replication. Aria ensures data consistency and fault tolerance through a shared log of updates. Replaying this log reconstructs the application state, simplifying replication and persistence.
The update log offers valuable debugging potential. In theory, stepping through log events allows for precise reconstruction of system states leading to errors. However, our existing tools for replaying Aria messages in Limshare were rudimentary. An interactive debugger allowed step-by-step update traversal with printouts of updates and some state information. Its primary limitation was the lack of snapshotting. Debugging sessions required replaying the entire message log from the beginning of the day to reach a specific time, making backward analysis impossible and iterative debugging cumbersome.
Alex Li’s internship project revolutionized debugging in Limshare by introducing a time-travel debugger leveraging Aria’s snapshotting capabilities. Snapshots capture the system state at specific points in time. Instead of replaying the entire log, the debugger can restore the system to the state of the latest snapshot preceding the target time and then replay only subsequent messages, drastically accelerating debugging. Alex also implemented the snapshotting mechanism within Limshare itself as part of his Jane Street internship.
The enhanced debugger significantly simplifies post-incident analysis. Consider investigating an unexpected order rejection. The debugger allows stepping forward to the rejection event and then stepping back to examine the system state just before the decision:
> step-time 10m (Decision (Resize_pool 2) REJECTED) stop_condition: Breakpoint: Reached a rejected request decision. current stream time: 2024-08-26 10:36:23.967645939
> back-messages 1 (Request.Resize_pool (pool_id 140737488355330) (downcrash $9_241_233) (upcrash $1_391)) current stream time: 2024-08-26 10:36:23.967491204Inspecting the state reveals limit usages and the rejected allocation request:
> print Checked out resources and limits
┌────────────────────┬─────────────┬──────────────┬─────────────┬──────────────┐
│ node id │ resources ↓ │ limit ↓ │ resources ↑ │ limit ↑ │
├────────────────────┼─────────────┼──────────────┼─────────────┼──────────────┤
│ kumquat │ U$6_453_178 │ U$10_000_000 │ U$34_748 │ U$10_000_000 │
└────────────────────┴─────────────┴──────────────┴─────────────┴──────────────┘
pools
┌─────────────────┬─────────────┬───────────────────────┬──────────────────────────┐
│ pool │ risk system │ request bundle │ size │
├─────────────────┼─────────────┼───────────────────────┼──────────────────────────┤
│ 140737488355330 │ nasdaq │ pts2, kumquat │ ↓ $0 │ ↑ $0 │
│ 140737488355329 │ nyse │ pts1, kumquat │ ↓ $6_453_178 │ ↑ $34_748 │
└─────────────────┴─────────────┴───────────────────────┴──────────────────────────┘
Undecided requests
┌───┬────────┬─────────────────┬─────────────────┬─────────────────────────┐
│ # │ Kind │ Node │ Pool │ Desired Size │
├───┼────────┼─────────────────┼─────────────────┼─────────────────────────┤
│ 1 │ Resize │ kumquat │ 140737488355330 │ ↓ $9_241_233 │ ↑ $1_391 │
└───┴────────┴─────────────────┴─────────────────┴─────────────────────────┘Analysis reveals that pts2‘s request was rejected due to pts1‘s existing large limit reservation. Further investigation, stepping back five minutes, shows the reservation’s duration:
> back-time 5m ("Enforcer lease " (has_lease_until (2024-08-26 10:31:28.713728082-04:00))) stop_condition: Time_limit current stream time: 2024-08-26 10:31:23.713892045
> print Checked out resources and limits
┌────────────────────┬─────────────┬──────────────┬─────────────┬──────────────┐
│ node id │ resources ↓ │ limit ↓ │ resources ↑ │ limit ↑ │
├────────────────────┼─────────────┼──────────────┼─────────────┼──────────────┤
│ kumquat │ U$6_453_178 │ U$10_000_000 │ U$34_748 │ U$10_000_000 │
└────────────────────┴─────────────┴──────────────┴─────────────┴──────────────┘
pools
┌─────────────────┬─────────────┬───────────────────────┬──────────────────────────┐
│ pool │ risk system │ request bundle │ size │
├─────────────────┼─────────────┼───────────────────────┼──────────────────────────┤
│ 140737488355330 │ nasdaq │ pts2, kumquat │ ↓ $0 │ ↑ $0 │
│ 140737488355329 │ nyse │ pts1, kumquat │ ↓ $6_453_178 │ ↑ $34_748 │
└─────────────────┴─────────────┴───────────────────────┴──────────────────────────┘The extended reservation duration, without justification, pointed to a bug. This example highlights the power of enhanced observability in simplifying debugging complex systems. Like our magic-trace and memtrace tools, the time-travel debugger makes previously hidden system behaviors visible, significantly accelerating issue resolution.
The debugger’s concept is broadly applicable beyond Limshare. The Aria team has adopted it, planning to integrate it across the Aria ecosystem. Several other teams are already interested in adopting it for their Aria applications, expanding the reach of this impactful project developed during a Jane Street internship.
Join the Innovation: Apply for Jane Street Internships
If these projects resonate with your interests, we encourage you to apply for Jane Street internships! Our internship program offers a unique opportunity to engage in real-world, impactful projects that will challenge and expand your engineering skills. As these examples illustrate, Jane Street internships provide a platform to make significant contributions to our technology and trading infrastructure.
Jane Street internships are designed to be immersive and rewarding, offering hands-on experience in a dynamic and intellectually stimulating environment. You’ll work alongside experienced engineers and traders, contributing to projects that are critical to our global operations. Whether you’re interested in programming languages, high-performance computing, financial systems, or debugging tools, Jane Street internships offer a diverse range of opportunities to match your skills and interests.
Take the next step in your career and explore the possibilities with Jane Street internships. Visit our internship page to learn more and apply. We look forward to receiving your application and potentially welcoming you to our next cohort of interns! Don’t miss the chance to contribute to impactful projects and grow your career through Jane Street internships.
Yaron Minsky, who joined Jane Street in 2002 and played a key role in the firm’s adoption of OCaml, authored this article. His insights reflect the deep technical engagement and innovative spirit fostered within Jane Street internships and the company as a whole.
