All Episodes
Listen in on Jane Street’s Ron Minsky as he has conversations with engineers working on everything from clock synchronization to reliable multicast, build systems to reconfigurable hardware. Get a peek at how Jane Street approaches problems, and how those ideas relate to tech more broadly.
Chloe Casale was planning to be an auto mechanic before taking an intro CS class on a bet. After a stint as a Forward Deployed Engineer at Palantir, Chloe came to Jane Street with no background in OCaml or finance. Now she leads a team of “desk devs” building tools for client-facing traders. In this episode, Ron and Chloe discuss how a surprising amount of Jane Street’s client business is conducted via chat, and the promise and perils of using LLMs to automate that trading. They dive deep on hiring and interviewing: does Jane Street’s process select for less “spiky” kinds of people than Palantir’s? Along the way, they marvel at Webs, an internal spreadsheet-like framework for composing computations and building UIs (which, you can see, is sort of hard to explain); compare building software for internal versus external customers; and recount the tale of Chloe’s epic 1,000km horse race across the Mongolian steppe.
Chloe Casale was planning to be an auto mechanic before taking an intro CS class on a bet. After a stint as a Forward Deployed Engineer at Palantir, Chloe came to Jane Street with no background in OCaml or finance. Now she leads a team of “desk devs” building tools for client-facing traders. In this episode, Ron and Chloe discuss how a surprising amount of Jane Street’s client business is conducted via chat, and the promise and perils of using LLMs to automate that trading. They dive deep on hiring and interviewing: does Jane Street’s process select for less “spiky” kinds of people than Palantir’s? Along the way, they marvel at Webs, an internal spreadsheet-like framework for composing computations and building UIs (which, you can see, is sort of hard to explain); compare building software for internal versus external customers; and recount the tale of Chloe’s epic 1,000km horse race across the Mongolian steppe.
Some links to topics that came up in the discussion:
So I’m pleased to introduce Chloe Casale, who’s an engineer here who works on building tools for our sales trading team. So first of all, I just want to thank you for joining me. And also, how’s your leg?
Thanks for having me. It’s still in a moon boot, so I got to get wheeled through the airport to come and get here today.
Tell us more how you broke your leg.
Yes. So I was doing this 10-day long endurance horse race in Mongolia called the Mongol Derby, which was a ton of fun. It was the hardest thing I’ve ever done in my life. But basically the premise is you fly out to Mongolia and then you have a thousand kilometers to cover over the course of this race. You swap horses, randomly draw horses every 35 to 40 kilometers. So by the end of it, you’re riding 29 horses, I want to say, over the course of the race.
Through the Mongolian steppe.
Through the Mongolian steppe. And you’re just given a series of navigation points on a GPS. So it’s not like there’s a trail. So you open your GPS and you make your best guess of the route. You get on a horse that’s been trained by people who live in Mongolia, which are very different riders than I am. And then you ride through the steppe in order to try to get to your next point in a reasonable amount of time because you have to finish on day 10 or else it doesn’t count as finishing the race.
So what bet did you lose?
I don’t even know what possessed me to do this thing. I think it’s like this piece of lore that I had heard was like, “Ooh, there’s this thing called the Mongol Derby and it’s really, really hard. It’s the longest, hardest horse race on the planet.” And I was like, “Ah, that sounds awesome. I do not have the resume for this thing.” And so one year they opened up the ability to go and test in basically to this race via the thing that they called the Academy. And in my mind I was like, “Yeah, I’ll just do the Academy. It’ll be a fun five days, whatever. I’ll come home, never think about this again.” Unfortunately, I finished the Academy and I got into the race.
So that’s the bet you lost.
Yeah. And I was like, “Oh, oh, okay. I think I have to do this now.”
So do you have a long history of horse racing before this?
Nope. I grew up around horses as a kid. When I was in university, I worked on a dude ranch during my summer. So I’d done horse things before, but it wasn’t like… I’m not a competitive rider. So this is why I was like, oh, I don’t think I’m going to get in. But I love backpacking in remote places. And so I think really the thing that got me about the Derby was the navigation bit where it’s like you have to navigate between points and you don’t know where you’re going and you might have to camp out on the steppe with a horse. And I just thought that sounded so cool, but it was very dangerous. I’m not sure I would recommend anybody do it who’s not prepared to risk life and limb.
And speaking of risking limb, you broke your leg?
Yeah.
How did that happen?
It was so silly. It was a stress fracture because when you’re racing, your feet are in stirrups and it doesn’t matter.
As I would expect, although I’m not sure what I would expect in the Mongolian steppe.
And so my stirrup for some reason was banging against my shin for the entire race. So I had a massive bruise. And so one morning on day nine, I wake up and I’m like, “Oh, I’m so smart. I’m going to shove socks up my pants to protect myself from the bruise I was getting.” Turns out that made it four times harder to bend my ankle, which is a very important joint to be able to bend in order to ride a horse. And so I got a stress fracture on my tibia from riding an entire day with literally just socks stuffed down my half chaps. And so I wake up on day 10, which is the last day of the race. I only have two stations left. So on a normal day, you’re going 90 kilometers. On this day, I only had 60 to go through and I couldn’t weight my leg.
You couldn’t put any weight on your leg?
No, could not put weight on my leg. So I was literally like, I didn’t even know what I was going to do, but I knew that if I could get on the horse, I was pretty sure I could finish the race. And so I just hopped to the horse line from my tent or from the yurt on the last day. And then they won’t let you ride unless you can get on the horse by yourself. And so then I just threw myself onto the horse. I got very lucky with the random draws on the last day. Very, very lovely horses. And then just managed to ride in the last 60 kilometers. I think I lost so many places in the race just because it took me so long because I had to go so slow because I was in so much pain, but I was going to finish that race.
I was like, I only got 60 kilometers left. I’ve already done 940 kilometers. I’m finishing this race. Yeah, but I finished. Only 23 people finished the race out of the 43 people who went to start camp.
Wow.
So it was a huge dropout rate.
Why are people dropping out mostly other than it’s insane?
I mean, it’s insane. I’d say that most people dropped out from heat exhaustion. This was the hottest year the race had ever run. And so you can’t ride a horse with heat exhaustion. And then after that was getting thrown from a horse and so getting breaking an arm or something.
And you managed to break a leg in a -
I broke a leg and didn’t even get thrown. I have no cool story. And then I think the last way was people getting kicked on the ground. But yeah, I mean, I was very proud of finishing, but it was definitely one of those things that you only ever need to do once, I think.
That sounds great. Just once.
Just once. It was over and never again.
So it’s anticlimactic to ask it, but I always like to ask about how people got into computer science. And now I feel like after learning about how you got into Mongolian horse racing, it’s hard to make that transition, but I’m going to do it anyway. How do you end up… Yeah, I don’t even know how to ask this question.
Why is a Mongolian horse racer at Jane Street?
Now I feel like we’re starting from the point of a Mongolian horse racer and then the ordering is all different from that in real life. But yeah, why are you a computer person? How did that happen?
I think these are just very entirely parallel hobbies of mine. I think on the one hand, I love doing these extreme sports. And on the other hand, I think I just really enjoy knowing how things work. I just love understanding how does the world around us actually function? And so when I was first in high school, university, figuring out what I wanted to do, I was convinced I was going to be probably a mechanical engineer. I wanted to work on cars.
Cars are cool.
Cars are so cool. They get us all over the place, and I had no idea how they worked. I think at some point I was like, I just want to know how these work and figure out how this thing that I feel like makes our world run actually works under the hood. At first, I thought I was going to be an auto mechanic.
Did you take mechanic courses in high school?
Yeah. I literally skipped chemistry in high school in order to go and take an auto tech class instead. I had to drive to a different high school, just a neighboring high school in my district in order to go take this class because they had the auto tech course. Got to go learn about how cars work, how do engines work, how do you change your brakes, how do you change your oil? If you ever need anything done on your car, I got you.
I do, but sadly it’s about the dumb computer stuff in the car, which is…
Honestly, surprisingly easy to fix. If you ever have the opportunity to go and watch someone work on a car, you’ll notice they have a computer, handheld computer that they have that they just go and plug in under your dashboard. There’s a slot, and then it just runs all the diagnostics for you and tells you exactly what to do on a lot of these modern cars. I’m convinced if I could just get my hands on one of these computers, I’d never have to go to the mechanic ever again.
Okay. But here you are not a mechanical engineer, although Jane Street also employs mechanical engineers, but that’s not what you do here.
No, that’s not what I do here.
How’d you end up in computer science?
I think my sophomore year of university, I was deep into my physics courses and I was hanging out with some friends who were all taking this introduction to computer science class. I just remember they were complaining about how hard this class was, and I said or thought, but probably said out loud, something along the lines of, “It can’t possibly be that hard.” My friends were like, “It is that hard.” And they bet me $20 that I couldn’t pass the class. I was a broke college student and I needed that 20 bucks, so I went and took this computer science course.
Was it really the 20 bucks or?
It was the 20 bucks and the pride.
I feel like maybe it was more the pride.
It was a little bit of both. I went and took this computer science class and I remember just all of those things that I got from learning how cars worked, I feel like I got in that class learning how to Java program, the simplest programs in the world, but being able to just type some things onto a computer and hit run and then have a game appear or something, I just remember thinking like, “Oh, this is magic. This is how computers work. This is really cool.” And then completely, I just fell in love with it. I knew that’s what I wanted to do. This was too late into my career at university really to change majors all the way to computer science because especially at University of Washington, they’ve got a pretty long CS course.
Yeah, they’ve got a really good CS department.
It’s very, very good department. I was not about to stay in university for five years in order to finish that course.
You were like Mongolian horses you had to do.
Yeah, I had places to be. I just pivoted my major to applied and computational mathematical sciences is what they called it at UW. It was basically an applied math course where you got to spend some time programming. I had a lot of the prereqs for that already sorted out and so I could graduate still in four years and get to do some computer science things.
I only studied math in undergrad. I actually did a little bit of CS on the side, but more theoretical math and applied math, but still even less relevant. Okay, so then what did you do from there?
So then I think when I was applying for jobs out of university, I kind of just cast a really wide net. I ended up getting a job as a forward deployed engineer actually at Palantir afterwards, moved to New York to go and do that. I was at Palantir for I think four and a half, almost five years and really loved doing a forward deployed engineer. I think that for me, it was another kind of one of those moments where it wasn’t entirely pure software engineering. I got to spend a bunch of my time thinking kind of like applied math. It’s like how do I apply this computer science knowledge that I’ve learned?
So what is a forward deployed engineer? This is, I think, Palantir jargon.
It’s Palantir jargon. It’s become weirdly popular in the industry these days, which I find interesting. But at the time, a forward deployed engineer at Palantir was basically there were two parallel engineering tracks, one forward deployed engineer, one product engineer. At Palantir, you called us a delta and a dev. I was a delta. I don’t remember why we called it that, but basically the idea was our job was to take whatever our product team had built and go onsite to customers and actually make it work for them in solving their specific problems. So it was a very specialized role in a way. I think sometimes when you think of software engineering, you think about trying to solve a problem for many, many customers, and a forward deployed engineer’s whole job was to solve a problem for one single customer. And so you got to go land on the ground, understand everyone’s problems very, very deeply, and then try to figure out, do I have to build something entirely new?
What products do I have that are supported by core Palantir that I can use in order to go and solve their problems? And then how am I actually going to make this valuable for the people who are there? And then you also had a responsibility of learning things that the customer wanted and feeding that back to product. So maybe when I’m working with a specific customer, I realize that, oh, this specific way of doing data analysis seems super powerful and that’s probably not just them who wants this. It’s like I get to go and feed this back to product and say, “Hey, I think this thing is generalizable. I think this is more interesting than maybe just this customer.” And then that’s how a lot of those roadmaps would get formed.
And do you feel like this kind of forward deployed role is something that just matched your interest and personality more? I feel like I know lots of engineers who are like, “You know what? I’d like to do anything that’s not that. I would like to do things that are building very general purpose as founded in systems and theory questions as possible and users, I don’t want to think about them.”
I think that’s the only thing I want to do. I think I’m the opposite dev to that dev where it’s like I just love talking to users. I think there have been various times as both a forward deployed engineer at Palantir, but also a software engineer at Jane Street where the easiest way to solve a problem was to send an email, and I’m more than happy to solve that problem that way. I don’t necessarily need to reach for coding every single time, and I think it just does, I don’t know, match my personality and the things I enjoy doing.
So how did you end up coming here from Palantir?
I think my time at Palantir was kind of coming to the end and I realized, okay, I just want to try something new. I’ve been here for a while and I didn’t feel like I was getting a lot of growth. I cast a just really wide net when I was interviewing. I was totally planning on probably ending up at another startup, but Jane Street kind of hit my radar and I was like, okay, might as well apply, might as well interview. I knew nothing about trading. I knew nothing about functional programming or OCaml. I knew all these things were things that Jane Street did.
We try and be very clear when talking to people that you can apply without knowing anything about trading or functional programming.
I was that person.
So, check, check.
I think that when I was going through the interviews, I just remember being so impressed with the interview process and being like, okay, people have clearly put a lot of thought into this. And by the time I was leaving Palantir, I was deeply involved in their interview process as well. So I was very much maybe an interview process snob at this point where I was very much like I had very strong opinions about the right way to hire. I think a lot of those opinions were probably wrong, but I think coming to Jane Street, I could tell that people had put a lot of thought into the way that you were interviewing here. And to me, that was a signal of a company that had it together and a company that cared deeply about its people. And that’s really what convinced me to join.
Yeah. I don’t know how good of a job we do, but we definitely put a lot of time and energy into it.
There’s a lot of energy going into it, and that was very apparent even as a candidate, even just things like having two interviewers in the room and every single one of my interviews, other than my P1, so other than my phone screen.
Yeah. I was actually surprised to learn how rare that is.
Super rare.
It’s almost always you’re being interviewed by one person, and we for a very long time have interviewed with two people. Part of the idea is that you walk out of the interview room and it just feels like it’s very useful to have another person who is there who you can say, “Wait, did you see what I saw? Maybe I was doing a bad job of driving the interview.” It just feels a very important way of both learning how the specific interview went, but also for people to over time improve how they run interviews. Getting that feedback seems really valuable.
Yeah. It’s super expensive though, right?
It’s super expensive.
I think that’s the thing. I remember just being so amazed of like, “Wow, we’re spending so much dev hours on this second interviewer.” But it’s the thing I think about every single season is are we getting the value from this thing? I think every single time I think about it, I’m like, “I think we still are.” It still does feel so valuable to be able to have that second pair of eyes, that second opinion.
Yeah, it’s clearly better, but that’s obviously the easy question. The question is not whether it’s better, but whether it’s worth it.
Yeah.
So one other thing, so you worked at Palantir in New York, but actually at Jane Street, you work in London.
Yes.
How did that happen?
I technically worked at Palantir in New York, but because I was a forward deployed engineer, I think I maybe spent four months in New York the entire time I was there.
Oh, wow.
So I worked in Brazil, Indonesia, Mexico, London, and Seattle when I was at Palantir.
This is another benefit of being a forward deployed engineer.
Yeah, you don’t have to have an apartment. Your entire life is out of one single suitcase, which was really fun when I was 20. I think I wouldn’t enjoy it as much now. I think at some point in my Palantir rotation, I ended up in London, and I just absolutely fell in love with London. I think it was the place for me. I always like to say I’m too soft for New York. New York is a very busy place. There’s a lot of energy here. It was really hard for me to go out and see trees.
I don’t understand people who don’t like New York. It’s loud, it’s sometimes kind of smelly, it’s incredibly crowded. What’s not to love?
I have no idea.
I really do love New York, but I do understand why people… It’s not everyone’s jam.
Yeah. And I think London was the perfect compromise for me. It was still a big city, but it wasn’t quite as intense as New York. And so when I was interviewing, I was like, “No, I’m going back to London.” So I tried to find a job that would land me in London.
Nice. Okay, let’s talk a little bit about what you do here. So you work on the software to help sales traders.
Yeah.
At a very basic level, let’s start, what’s a sales trader?
A sales trader is one of our traders. The way I think about it is they’re one of our traders who’s working with someone who they know who’s taking the other side of the trade maybe. So instead of trading anonymously on exchange, they’re trading with their clients, which maybe is a pension fund or maybe a big wealth management firm or something along those lines. A sales trader’s job is kind of twofold. It’s like on the one hand, they’re responsible for having this client relationship, understanding, oh, what are the kinds of things this client is going to want to trade with me? How do I facilitate that trade and making it come through Jane Street? Also, they’re responsible for helping actually execute those trades once they do get here. So thinking about not just capturing flow, but also getting the flow through our systems. So maybe that’s thinking about things like a client might reach out to us over a chat message directly to one of our sales traders and ask to trade something with us or request a quote.
And so our sales trader would be responsible for getting that into our systems, getting a price back from our prop traders, and then sending it back to the client.
Right. So maybe a bunch of things to unpack here. So first of all, what does Jane Street do anyway? Writ large, we’re mostly liquidity providers. Other people have desires to want to trade this or that, and mostly our role in the market is to sit in the middle and buy from one and sell to the other in lots of complicated ways that don’t just mean posting a bid and an offer on an electronic exchange. And one of the ways we do this is through these bilateral relationships with various other institutions that wants to trade through us rather than them just going directly to the markets. And you said this thing about prop traders, what’s a prop trader as opposed to a sales trader?
The way I think about it is our prop traders are responsible for understanding different asset classes, so a future or an equity or an ETF and understanding deeply how to price that thing. When I think about it from my point of view as a desk dev sitting with sales, it’s like our sales traders are the people who are trying to get in this bilateral flow, understand this flow, and then our prop traders are the people who are responsible for thinking about pricing this flow. Do we even want to take this flow on? Things like that.
And the prop traders in some sense, that’s the older role at Jane Street where you don’t have this kind of client oriented work and your primary job is building trading strategies, understanding the opportunities out there, thinking about actually are we making money doing this or not? Those kinds of issues. And sometimes thinking directly in a very detailed moment by moment way about individual trades, sometimes building software that executes trading strategies, lots of different pieces of that. And the sales trader on the other hand is much more about understanding the relationship with a client. And why does a client want to have a relationship with us? Why don’t they just want to trade on exchange? What’s the value add that you get from hitting us up on some chat platform and trading with Jane Street that way?
I think maybe the easiest way to think about it is a lot of these clients, they want to do very, very large trades, trades that maybe just don’t make sense to actually execute on the exchange because you’re not going to be able to get the best price. The simplest example is you can imagine that maybe I am rebalancing my pension fund and I want to be able to put through a bunch of very large orders for very specific ETFs. Maybe right now the open book on the exchange doesn’t have enough sell orders for them to actually go through and basically execute this whole trade.
That’s right. There’s all this big continuous electronic system where people are constantly posting orders, bids and offers, the willingness to buy or sell a given security for a certain size or less for a specific price or better. And there’s actually just a limited amount of liquidity that’s out there at any one time. There’s this fundamental adverse selection problem, which is if you’re a prop trader and you have a system that posts orders all the time, a lot of the time someone will come to trade with you who’s just someone who wants to trade in or out of something. And sometimes someone will come to trade with you specifically because they realize that the pricing in the market is off and they’re trading with you conditional on the fact that they think your prices are wrong. And so when you trade with those people, you lose a bunch of money every time, and then you make a little bit of money just kind of collecting the bid ask spread or half the bid ask spread every time you do a trade for everyone else.
And so it means that people post less size than they otherwise would, and they post wider spreads. They give out worse prices than they would because they kind of have to make the money back from all the kind of ordinary back and forth facilitation trades to make up for the fact that occasionally someone comes who knows a lot more than the market maker does.
Yeah.
Right. And then the dynamic is very different in a client.
Yeah, because in a client in a way, it’s like I have information about why you’re coming to trade with me. I think sometimes when you’re trading anonymously on exchange, it’s to your point, you don’t necessarily know what the person you’re trading with knows, but when you’re trading with a client, it’s very frequently, I know exactly why you’re coming to trade. I’m not super worried about you having a bunch of information and coming to pick me off because I know why it is you want to do this trade. And so for that reason, we can give better prices than going straight to the exchange and trying to trade there.
Right. And a lot of this also leverages the fact that we are really good traders of a bunch of different kinds of derivatives. There might be an ETF, an exchange traded fund where you have a single security that is a proxy for a basket of things underlyings that are themselves maybe much more liquidly traded, but not that many people trade the particular ETF, so it’s not that thickly quoted, so it’s hard to get a big order off there. But if you go to us and say, “Oh, I have a really big order to do,” we can do a good job of pricing the ETF off of the underlyings and then we can unwind the trade on our side. We buy to you or we sell to you, we can do the other half of that of getting rid of that risk in the underlyings if there’s not enough liquidity in the ETF.
And that’s another part of the dynamic.
And that’s a huge whole process that a lot of these companies, they’re not going to want to try to figure out how to build in-house.
That’s right. Part of what you’re doing is there’s a kind of simplicity. If you can just go to someone and be like, “I want to do this big trade. Can you figure it out?” We’ll get some extra fee implicit in the prices for it, but there’s a huge amount of expertise they don’t have to develop.
Yes. Yeah.
And then I guess another thing is why chat? It sounds like -
That really surprised me when I joined Jane Street. I think when someone was like, “Oh yeah, people are still sending me chat messages or sometimes even picking up the phone,” I was like, “Whoa.” But I think maybe the simplest version of why chat is you can imagine maybe the alternative is some sort of electronic system where you could go in and type in maybe a request for quote is a good example, an RFQ. So someone is requesting a quote maybe for buying a hundred Apple or something.
And when you say requesting a quote, what does it mean to request a quote? That’s also jargon.
Yeah, so I guess if I was to come to you and basically you’re saying if I wanted to buy a hundred shares of Apple, how much would you sell that to me for? And that’s the quote. So you’re going to give me back a price, probably per share, and then I’m going to know whether or not I basically want to take that offer that you just gave me. And so that is a request for quote.
Right. And someone who’s coming to trade might send out an RFQ to many different liquidity providers at the same time and then pick the best price from those.
Yeah. Yeah. And so you could imagine doing this process over what we call RFQ platforms. So those are companies, systems where you can go and just enter it into roughly a form and then hit enter and then it gets sent electronically. And so it’s a very standardized protocol. It’s lovely to work with, especially when I start thinking about chats. But these platforms, it’s kind of hard to onboard to them. So it’s like you have to build an entire system internally that can have communication with these platforms. At Jane Street, we have a whole team who’s responsible for thinking about integrating with RFQ platforms, and there’s a cost associated with onboarding to them. And so in some ways, especially when we have these clients who are maybe doing very, very few trades a year, it doesn’t make sense for them to build out all of this integration with these RFQ platforms and doesn’t make sense to pay for the cost of this thing when you can just send a chat across Bloomberg Terminal.
Although I would’ve thought we have to do a lot of integration work because we want to hook a bunch of automated systems in, but I would’ve thought that an end client would have to, I don’t know, open up their web browser and go to a URL and pull up the RFQ platform that way. Why is it more complicated than that?
I think sometimes it is more complicated than that because you actually do want to integrate so that you can get the standardized protocol. I think sometimes it’s not more complicated than that. You do just need to install the software, but you do have to pay for the software, and that’s kind of paying for something on top of maybe just a Bloomberg contract.
And a thing that maybe you’re not going to use very often, so it’s paying for itself.
It doesn’t make sense.
I think also sometimes these platforms don’t have the exact kind of flexibility that clients want. I think that a simple thing to imagine is sometimes a client wants to send off a request for quote, but not include the size. Maybe this is because they don’t know the size yet, maybe this is because they don’t want us to know the size yet, but some platforms don’t allow you to send an RFQ without a size field, and so they would have to either find a platform that allows them to handle this kind of custom flow, is the way I call it, or they can just send a message over chat.
And also, I guess chat is just older.
It’s older.
It hasn’t been competed out, right?
Yes. Maybe one day.
So how do we deal with chat? That’s obviously kind of messy. Humans are typing in messages and then agreeing to trades, which are maybe quite large. As you pointed out, some of the reasons people come to us is they’re very large trades. That sounds a little error prone and hard to automate. So how do we deal with that?
It is hard to automate. I think that, especially going back to pre-LLM times, when we were thinking about automating chats, it’s like some of our chats were relatively easy to automate because you could tell that someone was putting in a standardized format every single time they were sending us one of these requests for quote. So it was relatively easy for us to build a system that could read the regex, could use a regex parser, parse out this request for quote, and then send it through our RFQ systems. And this felt relatively low risk. It was like we had very consistent messages coming from clients. We still had a sales trader looking at the chat room every single time, even though the parsing was happening automatically, a sales trader would still go back into the chat room in order to maybe go and send the price that we ended up quoting.
And so we thought a lot about just the automation of getting this into our systems and understanding, okay, what’s good, reasonable regex parsing here? I think a lot of doors feel like they’ve opened since all of the access to LLMs that we have. I think now instead you could imagine a world where instead of having to write some specialized regex parser for every single counterparty we’re trading with, you could just maybe write a specialized prompt, maybe just one prompt for one asset class. There’s so many things you can start thinking about here in terms of how can we start parsing these requests from clients out of chat into a more standardized ways for our traders to look at.
So even in the old regular expression world, it’s maybe worth going through what’s the actual value add? If at the end of the day there’s a sales trader who’s looking at the chat and thinking about it and typing in the final price, actually, why do we care about capturing the data? I mean, the trader is there, they can see it with their own eyes. What’s the benefit that we get by this kind of automatic extraction?
I think some of it just is speed. I think that some clients do care about speed. We are racing in a way with some other brokers and we want to win on price, but sometimes people just really want to get their orders through very, very quickly. I think there is also a thing here where when you have very standardized parsing, it’s like you can be pretty confident that you’re not going to have a big fat finger mistake. And so even though we have a human double checking to say, “Oh, okay, these two things are the same,” it’s like a human quickly looking at your parsed RFQ and your chat message is faster, and I might even argue less error-prone than a human trying to figure out how to exactly type that thing over into our internal system.
Sure. In fact, I would expect it to not be close. I would expect you’d be much more likely to make mistakes in the hand entering of things. It’s really easy to hit one digit in the wrong place, put the decimal point in the wrong spot.
Yeah.
Can you give me a little more detail? I’m just trying to understand why would the speed matter to the end user there? What are they doing? At what timescale does speed matter?
Oh, this matters, I’d say different counterparties have different amounts of speed, and it’s also interesting because it’s actually quite different every region, I’d say, how much people are sensitive to speed. And so you have some counterparties who we are aiming really to try to get quotes back, at least are trying to get quotes into our system within less than 20 seconds. Some counterparties were trying to get it in less than five so that we can basically have the time for it to go get a price, be able to write that price back into a chat. So this is still human level processing, but I think if we started taking, I don’t know, 45 seconds on something where someone is expecting us to be back in five, we might lose that trade. That being said, there are some clients who are sending us trades that they’re expecting a price back within an hour, and so it’s very different people are executing different strategies on their side is the way I think about it.
I think some counterparties might be thinking about trades that are maybe a little bit more time sensitive while others are thinking about things that are maybe closer to trades where you’re striking the price on the close or something.
Yeah. I wonder how much is it the actual time sensitivity of the trade they want to do and how much of it is like, “Well, I want to do this one. Well, it’s really annoying to have to wait 45 seconds for the next. So I’m going to wait a little while and see what comes back and I’ll choose among those,” and just like a workflow issue.
Yeah. I mean, there is a human on the other side and it’s really hard, I think in some ways on our side to guess necessarily as to why different people… You can take some guesses as to why, oh, maybe they might be more time sensitive versus this other counterparty. But at the same time, you’re right. It might just be the human on the other side kind of got bored waiting. It might be the human on the other side wants to kick off their next trade. They don’t want to have to be thinking about five, 10 live trades all at the same time. There’s so many reasons why someone might want to say, “I’m ready to execute this thing.”
I’m sure you spend a lot of time with our sales traders trying to understand their workflows and what’s important to them.
Yeah.
How much time do you end up spending thinking about what the actual clients want and how that works and what we can learn from that?
I spend a lot of time figuring out how I can facilitate our traders thinking about what their clients want. So I think that they have the expertise of understanding. They’re talking to these clients very frequently. They’re seeing the flow go by. They have hypotheses as to what’s going on. The extent I start thinking about our clients is thinking about, oh, what kind of questions do my sales traders want to be asking in order to figure out how to support their clients and then building systems that can facilitate that question and answering.
Okay, so now we’re in this new large language model world. How does that change the parsing story?
It’s kind of opened the doors to being able to parse things that are maybe coming through in a less structured way. So it’s like counterparties who are actually typing things in by hand every single time and maybe are doing different formats, maybe they have very typos. You can imagine we can start handling those things. At the same time, LLMs, they’re non-deterministic. I think that I get nervous about what things that they’re going to get right or wrong, and I think it’s kind of hard to tell how comfortable we should be even inserting them into a process. And a thing that I think about very frequently is we very frequently talk about, okay, well we can put a human in a loop and then a human can give us some sort of fact checking or some sort of saying, yes, this chat message looks like this parse.
But when you’re thinking about when does that work? If your parser is right, I don’t know, 95% of the time, a human is kind of unlikely to notice that 5% of the time in which it was wrong because they’re just so used to being correct, they’re just constantly kind of hitting the yes button. This is just human nature.
Alert fatigue.
Yeah. If the thing is right 25% of the time, the trader’s just never going to look at it because it’s always wrong. So thinking really hard about, okay, these LLMs, I’m not going to make them artificially incorrect. That feels bad. But also, okay, so let’s say they’re probably going to be right 95% of the time. When they’re wrong, 5% of the time, let’s say, what kind of risk checks can we put in place? What kind of things can we do with our system in order to make us feel protected about what that 5% might look like?
I mean, you could literally insert errors, not make the LLM make a mistake, but insert errors into the confirms at some rate and see whether people catch it. And when they don’t, you’re like, actually that was wrong.
Yeah, so this has crossed my mind.
I’m not sure how much the sales traders would enjoy this process.
Yeah, I pitched it and I think the sales traders were a little less than happy. And I think at the same time, it’s hard to figure out how much is that actually going to cause people to actually, I don’t know, bring up that extra scrutiny level. I think -
And you could tell how good they were at it.
Yeah.
It’s tempting.
It’s tempting. It’s tempting, but I think that instead, I think spending some time thinking about how do I catch those 5%? How do I get this closer to maybe 99%? How do I make sure that when the 5% goes wrong, it’s at least easy for someone to notice in a way. It’s like, can we have two different prompts parsing the same chat message? Do these things agree? Do they disagree? There’s all sort of ways that we can get creative to get ourselves a little bit more confident at least catching those 5% that isn’t necessarily relying on a person.
Right. And obviously you’re in a world where the underlying technology is changing really fast. It’s also maybe not always changing in a way that’s good for what we want in the sense that one thing that happens now with models is they spend a lot more time chewing on and thinking about things, and you actually probably want them to be able to turn around and respond pretty quickly.
Very fast. Yeah. So we’ve been even thinking about some of our counterparties, they just can’t move on to LLM parsing because they just want their responses too quickly and it’s very hard to guarantee how fast that response is going to come from versus other counterparties, maybe it’s okay to have the LLM deep reasoning in order to get the exact symbol that this counterparty has asked for.
Right. And there’s a question of do you give the LLM just the one chat message or the history of chat messages where it might in fact be better able to build priors if it sees more stuff, but it’s of course going to take longer to do its inference in that world. Have you thought about, there are all these fancy fast chips for doing much faster LLM inference that can get a factor of 10 or something faster inference at much higher cost?
I think we’ve kind of bounced around the idea. I think that right now I’d say that even our LLM parsing just feels like it’s very, very first stage. I don’t think we’re at the point where we’re trying to figure out how to optimize to get these things through very, very fast. I think we’re still at the part of trying to fine tune our prompts so that even when they’re thinking for a very long time, we’re getting the right thing out. I think that maybe at some point we’ll get there to this level of optimization, but I do think one of the nice things I think about working in client-facing tech is we are always kind of working on these human scales. And so I think that we do have some escape hatches.
And then when the LLM is wrong, what happens? What are the implications of mistakes in this context?
It’s really complicated. You can imagine that the LLM… Let’s imagine a worst case scenario where the LLM is wrong and no one notices that our RFQ maybe got parsed completely incorrectly. We get a price back, maybe I asked you for a quote of Apple and instead I’m quoting Tesla, whatever. We get a price back and then maybe someone just copy and pastes the price in no context back into the chat room. I don’t think that’s going to happen, but hypothetical world, it’s like then there’s kind of this idea of there’s been a meeting of the minds over chat where it’s like we’ve kind of decided together that this is the request for quote they asked for, this is the price that we gave them. Someone on their side says done. And then someone on our side says done, and then we need to go and execute that trade.
And we are working with clients, so they’re not necessarily adversarial. You can imagine a world where we send them some crazy price back and someone’s going to respond and just be like, “Really?” And so there are moments where someone might notice that something looks a little bit wrong, but it is possible that we send back the wrong price and we’ve agreed to the wrong price and then we make a trade on a price that we didn’t necessarily want to trade at. And there’s relationship cost for going to a client and saying, “Hey, I know I agreed to this trade with you, but we actually can’t do it.” And that’s a thing that a sales trader has to be thinking about when we maybe run into a mistake like this is what is the cost of going back to this counterparty and saying, “We can’t do this.
Is it too heavy? Should we just take the loss on this trade? How are we thinking about this?”
Right. One thing that’s worth noting is that even when both sides say done, which is a weird bit of trading jargon for now the trade is settled, it doesn’t actually mean the trade is done.
No.
It’s just to say if it’s just obviously a screw up, then people will go back and be like, “Oh no, that’s not what we meant,” and fix it.
Maybe.
Maybe.
Maybe. I think it just kind of depends on what the cost of doing that is because you could imagine that I go back, maybe I noticed the mistake, I don’t know, 30 minutes later and now the price has moved. There’s a whole thing here of we have made a mistake in this scenario and we have to ask ourselves what is our obligation to our counterparties in that moment.
That’s right. And there are relationship questions there and you don’t want to impose weird things on your clients and things like that. On the other hand, if it’s like the price is off by a factor of a hundred, which a hundred sounds like a crazy amount for it to be off, but it’s actually a totally ordinary kind of mistake. It’s like, was that meant to be quoted in pence or pounds?
Yes.
And you could just be off by a factor of a hundred and everyone could be very confused. But then in that world, everyone understands it’s erroneous. And your use of the word meeting of the minds is like, that’s not accidental terminology. Part of what’s going on is you’re forming a contract here.
Yes, we’re forming a contract.
Whether or not there is a meeting of the minds is kind of part of the legal standard of whether that’s the real thing or not.
Yeah. It’s like whether or not we’ve kind of made this trade.
So are there things that we can do on the technical side to reduce the chance of people being confused even when we have a misparse?
For sure. So I think one of the things that we think about is giving our traders specific strings that they can copy and paste back into that chat message that are non-ambiguous. So let’s say someone gets a price back and instead of just sending the price, it’s like we can say exactly what it is we’re going to do like, “Hey, this is our price for buying a hundred shares of Apple at this settlement date,” and then send that through exactly. And so this means that even though maybe our counterparty didn’t read it, there are still obviously human mistakes that can be made. It gives us all a better opportunity for everyone to catch the mistake as it’s being made and our external counterparties to be able to say, oh, okay, I see this thing.
So if you’ve gotten it wrong, we’re at least explicit.
We’re at least explicit in what we did wrong.
Sometimes we put in the same information into the chat that we use in calculating the price that we gave them. So even if we calculated the price for the wrong thing, we’ve at least told them that we’ve done the wrong thing. And that part’s not some text generated by an LLM, that’s some hard system that understands what’s being done and has an explicit transformation to present that in a way that is unambiguous.
Yeah. And we work with our traders on defining what is unambiguous in this context. I think as a dev, I don’t have necessarily the context of knowing what an unambiguous trade is, but when we go and have conversations with our traders, I can say what exactly needs to be in this string so that everyone knows what it is we’re agreeing to.
That’s right. And you might think, oh, it’s the name of the security and the price and the size. But you mentioned settlement date, that’s one example of a weird thing of when do the actual shares really change hands and that can affect what’s going on. What other weird things do you sometimes need to stick in there?
Oh, I don’t even know. I think options are particularly interesting, especially when you start trading complex options. It’s like you have to -
Where it’s like a single trade in a basket of multiple options and maybe multiple options and the underlying.
Yeah, exactly. And so those symbols can look crazy. And so being able to understand exactly what kind of symbol needs to go through, where exactly the plus and minuses signs go.
Oh right. There’s horrible quoting conventions of the price could be positive or negative depending on how you try to decide to quote the basket.
Yes. And then you also have to think about all sorts of things I barely understand, but it’s like quoting things maybe using delta risk versus some other kind of risk. It’s all these things you have to be thinking about when quoting options. And so the fact that I have an expert that I can go and talk to makes my job much, much easier.
Awesome. Okay, so chat parsing is obviously an important thing, but you’re also building tools for people and building tools for people involves building user interfaces.
Yes.
So say a little bit more of, as an engineer at Jane Street, what are the options in front of you of how you might build a UI for a sales trader?
Yeah, so I think that there’s a couple of things. I think one way that you might build a UI for a sales trader is through our, do we call Bonsai a framework?
I guess, yeah. It’s like a web framework, whatever you call React.
Yeah. So our internal version of React, I guess, which we call Bonsai, that feels a little bit mean to Bonsai, honestly. I think that Bonsai is a super powerful way of building front ends in OCaml. Also, I’d say is an expert way of building front ends. I think that in order to build a front end in Bonsai, you have to know Bonsai and you have to know OCaml and you have to be willing to put in the time to build a proper UI. Sometimes when you’re working with a sales trader on the desk, you don’t necessarily need an entire proper UI that’s built out. Maybe you just need something that gives someone a quick view of their data or maybe just a quick dashboard that they’re only going to need for four weeks and then we can tear it down. And so another way that you can build a UI for our sales traders or a lot of people at Jane Street is something that we call internally Webs.
Webs is exceptionally difficult to describe because I think you can think about it kind of as a distributed computational graph maybe.
I think that’s fair. That’s one of the things it is.
You could also think about it like everything Excel wanted to be. It’s like this amazingly powerful tool where you can build what I would call UIs where you can display data, usually tabular data, but so much more than that. I’ve seen people iframe in websites into Webs before, and so it’s really possible to create these very powerful UIs in Webs.
And it’s maybe worth saying, this is a weird kind of technology that shows up in some form or another in lots of different trading firms. So if Goldman Sachs famously has a system called Slang, Slang is their programming language, and there’s some underlying data store called SecDB, securities database, where they build all sorts of applications on top of it. And a thing that you mentioned graph, why is there a graph there? A thing that has shown up for a long time in trading systems is you have streaming data that’s coming in and then you want to compute something from that streaming data. But the streaming data, everything doesn’t change all at once. Little parts of it change, and then you don’t want to have to do all the computational work of recomputing all of it all at the same time. And so you have some kind of graph that gives you some structure of the computation, so only the parts that need to be recomputed actually are.
It’s like a form of incremental computation. And also spreadsheets totally do this, right?
Yes.
Spreadsheets are this fun functional programming language. Every cell is a little purely functional expression. It expresses dependencies on other cells that induces a computation graph. And Webs literally developed from what was just an overgrown spreadsheet. We had big spreadsheets on one of the desks that were used for pricing a bunch of their securities and Webs kind of, in fact, that spreadsheet was called Webs eventually, and then we’ve built a system in the image of that spreadsheet, which takes that same basic structure and kind of industrializes it and makes it bigger and more scalable. And maybe one of the things that’s different from what you would normally expect from a traditional spreadsheet is it’s much more of a collaboration tool. The big computational graph is shared among a whole bunch of people. And so it’s both a way of building some new computation and it’s also like a playground where all the other computations that other people have done are just sitting there available for you.
And then it has a kind of built-in little GUI builder in it, an easy way of building views. Traditional spreadsheets have this very flat grid view that you can embed all sorts of things in. But no, this is like, even though there’s lots of tabular data, you have a little programming language for constructing views as part of Webs.
Yeah. And I say this programming language is very approachable, which I think is one of the powerful things about Webs is it’s very easy to basically maybe take someone who’s a little bit less technical of a user and give them the basics of how to build a new view in Webs and have them go up and be able to create a view that it is exactly what they want. And I think to one of your earlier points, Webs is this, it allows you to share so much analysis, but it also allows you to share data. So once you have one piece of data in Webs, anybody else who’s using that Webs instance can go and basically access that piece of data in order to run whatever analysis that it is that they want to do. And so when we’re spinning up a new Webs instance to actually think about using in recruiting, I think one of the conversations we have with a lot of our recruiters is Webs is more powerful the more brave you are.
It’s like the more frequently you’re willing to go and try to build a new view, other people can build on top of that view that you’ve built. And these things, they really compound in terms of I added something and then you added something and then this other person added something and all of a sudden you have this UI that’s way more powerful than it would’ve been if everyone was too afraid to touch Webs.
Oh, that’s really cool. So is recruiting on board or are you guys starting to actually do this?
Yeah, we are. Yeah. So recruiting’s on board. We’ve got a couple of views that are in Webs now and we are starting to do this. And I think it’s been really cool to watch, especially Webs has, and its UI has a really great integration with AI. And so it’s like you can basically hit a hot key and then you’ll have a little code assistant basically appear, and then you can have a chat with your AI and say, “Hey, this is the thing that it is that I want to build.” You’ll get a Webs view from that. And this has been really great for onboarding a lot of our recruiters to being able to write their own Webs views of the data analysis that they want to be able to run and so that they can dig into the data that they’ve had access to without necessarily needing a whole dev to get involved in order to pull that data for them.
Great. So you get to empower the people more directly, just build the things that they want.
Yeah.
And I guess another nice thing about Webs is it’s not just a language and a runtime for this, but also a deployment story. Just like with Excel, you just go and put stuff in Excel and when does that start working? Immediately.
Now. Yeah.
That’s right. And it’s sort of the same thing if you add a view in Webs, it just locks in and hooks into the production system right there and you can immediately extend it.
Yeah.
Okay, so we can do full-on web applications in our own super weird alternate world, OCaml-based web programming thing. You can do more or less the same thing in this widely shared graph of data and computation thing called Webs with its own also super weird custom programming language that we created.
Yep.
What other options are there?
I think the biggest other thing that at this point I see people doing is just going and vibe coding up just a UI.
Just an index.html with some JavaScript right in there.
And some JavaScript that they just push up to a location where everyone on the desk can see and people just can start using it basically immediately. And I’d say that I kind of said these things in that order because I think that this is kind of the way that I think about maybe productionizing something.
Although in the opposite direction.
In the opposite direction, yeah. Where it’s like you’ll have a trader maybe have an idea and they’ll very quickly vibe code something up just to maybe see what it is. And then we can very quickly iterate on, okay, how valuable is this thing? How many people on the desk are using this thing? Does this thing need to be productionized or is it just a read-only thing that doesn’t need a bunch of dev scrutiny? And then from there, it’s like we can start having that conversation of like, oh, maybe this thing could live in Webs and continue to make Webs more powerful to my point earlier. Or maybe no, we think this thing is going to be very critical for a bunch of things in trading, but also we need a very custom UI in order to make it very easy for people to interact with.
And that’s kind of the point in which we would start moving over into a Bonsai UI.
So obviously AI is inflecting all of the different choices here. And in fact, you can use AI tools. We sort of mentioned it for the latter two options, but for option number one, you could also have an AI write a Bonsai app for you and the skills and tools around that are getting better all the time. How do you think about the safety trade-offs of having, especially less technical users who maybe don’t understand the system that well and maybe aren’t really coders at all, use this stuff and have LLMs generating things for them that they actually don’t understand and that nobody understands and then no human has read?
Yeah. I think especially on the sales desk, a thing that we’ve been thinking about a lot is how do we think about varying our scrutiny? And I think this has always been a skill that we expect devs to have. It’s like some things need very, very, very high scrutiny and you really need to stare at it in order to believe it’s working exactly the right way. And maybe some things you can maybe have a little bit less scrutiny on because maybe this thing is not super very important or maybe this thing is only going to be used once for something that maybe doesn’t super matter. And I think when we’ve been thinking a lot about vibe coding our apps on sales, a thing that we’ve been thinking about is what are things that feel very safe to do with a system that we don’t fully understand and what kinds of things feel not safe to do with a system that we don’t fully understand? And so maybe an example of one is we want to pull some data to give ourselves a sense check about whether or not any clients that previously were trading with us have stopped trading with us. So this is a read-only dashboard. It’s not going to cause any trading decisions to change. We’re not going to go and publish this data to any regulators.
All sorts of things about this make this feel like a very easy thing to vibe code up. But if any of those things that I just listed were not true, it’s like now all of a sudden it’s like we really do want someone to start thinking about the code and start having some more scrutiny on, okay, do we understand exactly what this thing is doing? Is this pulling data in the correct way?
But then how do you actually deal with knowing what it can do if no one has read the code?
I’ve never really considered that someone has built up a dashboard that was then doing something very completely unexpectedly, trying to send a request off to a database that it wasn’t prompted to do. I guess it could do this thing.
I mean, if Hugging Face is any example, there’s lots of… I actually do think there are real things to worry about there. In part, even if you ignore the model with its cyber capabilities, guards turned off, running wild and hacking into things, the model’s trying to be helpful and not having a lot of context and knowing what’s actually reasonable to do and what’s not reasonable to do. And the model doesn’t necessarily know what’s reasonable to do and not reasonable to do, and the person driving the model maybe doesn’t know either. So one thing that’s nice about the index.html thing is it’s actually, the whole world has already been built up under the assumption that you can’t trust webpages. So there’s sandboxing built in and limitations there. And then when it does reach out over the network, we can actually can and put in various kind of controls over what calls do you make available to any comer under the assumption that a random webpage is actually not fully trustworthy, even if it is authenticated as the person who’s doing it, maybe the person who’s logged into that webpage isn’t even the person who wrote it and you don’t want to misuse their credentials.
And so we have tools in that place. But then as you think about, but what you throw into a Webs instance, well, anything that Webs instance can do, you’re maybe worried about that. And if you have someone vibe coding just a full-on program for something, you have more problems of that shape.
It’s problems that at least I feel like I don’t have good answers to at this point. I think that it very much feels like you’re trying to keep up with all of these advances in AI. And I think especially as a dev, it’s like I don’t want to slow down all my users. They have access to these really powerful tools and they can get things done that they weren’t able to get done maybe even two months ago. But also some of the things I do find kind of scary, and I don’t know, I think it’s one of the things that I’ve been thinking about a lot, even as simple as we have a lot more code review to do as devs these days. And so there’s a thing where it’s like more code is just being generated by AI. We have more code review to do.
I don’t know about you, but sometimes I get a little tired. I think there’s a thing where how do I keep my scrutiny up on the code review, especially when I’m code reviewing code that maybe I’m the first dev to have looked at. I think it’s all really hard.
Yeah. I mean, there’s a lot of things that we’re thinking about here and doing. There’s a lot of work on sandboxing of agents to help deal with this kind of issue. An idea that I’ve been thinking about just in the last couple of weeks is to give safe pathways for vibe coding whole applications where you don’t have good sandboxing at the application layer by doing stuff where you have a separate document that specifies in some sense what the guards are supposed to be, and then maybe you could have hard semantic checks. You could say it’s like, “Oh, this document is there, so we have to build this in a way that there’ll be sandboxing built in that will enforce the rules that are stated there.” Maybe simpler and easier to implement, although having its own limitations of having mandatory LLM review so that when you make a change, you have another…
The LLM that you’re trying to work with that is trying to solve your problems that might do creative and problematic things. But then you have another separate session whose only goal is to look over the diff, look over the spec, and to try and make sure that the spec isn’t being violated so that even if a human isn’t checking that, this very limited form of verification that you need the LLM to do, maybe it can do a good enough job of that that you can prevent people from making mistakes. And then that little file that specifies the bounds, that you would actually have to have a person review.
Yeah. We kind of have a similar workflow that we spun up on sales where when a person goes and vibe codes out their app, they then go and build something, basically a document that explains what it is that they think their app is doing. And then we have a skill that runs on both the app and the document that lets them know how much scrutiny we think should be applied to this thing. And it will say, “Oh, hey, I think you should pull in a dev,” or, “I think this thing is fine for it to just get spun up under vibe coded.” And so we have seen some adoption of that, which I think has been actually quite useful because it forces someone to take their app, go and think about what are the actual requirements of the thing I’m building, what is it actually doing?
And then has an LLM go and say, “Hey, I think you should ask a dev,” or, “I think this is super safe and I wouldn’t worry about it.” But I think having some LLM jump in has been nice there, especially because it’s hard for us to read every single app that’s getting vibe coded out. But having a skill that has been a little bit trained on how to think about these things has been nice. It doesn’t get it right all the time. It might be overly conservative, but I think I prefer it that way.
Yeah, no, I think the technological choices here are really hard to work through. The other thing that we’re actually just starting to do is we’ve started hiring a team of people to work on formal methods so that you could actually prove properties of your program in a way that gets you a much higher level of confidence that the conclusions are right. And I think that will also be useful in some places, but actually I think the trade-offs here are actually really hard to figure out. And we’re trying to figure out things that we can build that will work across a range of possible future model capabilities because the future is very hard to handicap. Models could vary quite a lot in how strong they are in a year or two or five.
Yeah. Even just watching the advances in the past one year, it’s crazy.
It’s really weird. Yeah, no, our world is changing really fast. You mentioned before that you’re a desk dev, and at Palantir, you were an FDE, a forward deployed engineer. Those sound kind of similar. Desk dev is the term that we use for basically a team that is embedded with a trading team of one kind or another. I’m curious, you’ve now seen similar shaped roles here and somewhere else, and how do they differ? I feel like what is it like in different places doing this kind of deeply embedded role?
Yeah, I think the biggest difference between doing the FDE thing at Palantir and I think being a desk dev here is your incentives are just very aligned with your customers in a way. Here at Jane Street, my customers are the traders that it is that I work with. The thing that they’re trying to optimize for is Jane Street, and the thing that I’m trying to optimize for is Jane Street. And so there’s this thing where our incentives are very aligned. Having all sorts of conversations with traders, I find so much easier than I did with customers because you can have these very open and honest and frank conversations with a trader at Jane Street about things that are easy, things that are hard, things that I think are going to take a lot of time versus a little bit of time. And you can prioritize things together in a way that when you’re working with an external customer, part of my job at Palantir is to sell Palantir.
The job of the external customer is not to buy Palantir, it’s to do their job the best they can do. And so I would have to think a lot harder about, okay, well, what kinds of things am I bringing to a customer? What kinds of things do I want to present as easy? Are those things actually easy? Wasn’t necessarily always true. And having those incentives a little bit misaligned, I think just made that job a little bit harder. I think that being said, it’s like when I was working at Palantir, I cared very, very deeply about solving my customer’s problems. And so I think that piece I think feels very similar. It’s like going and understanding someone’s workflow very, very deeply to the point where I feel like I can actually go and make it better, I think feels very similar. I think here, all the conversations about trade-offs and prioritization and how exactly we’re going to accomplish this thing just feel a lot easier, a lot more upfront.
Are there any things about how Palantir structured the forward deployed story that seem different? There’s obviously you’re pointing out ways in which it’s just structurally different. The incentive structures are different, but putting that part aside for a second, are there other ways in which it was handled in a materially different way than what we do? And is there anything for us to learn from that?
One of the pieces of forward deployed and the way that it worked was as a forward deployed engineer, you were expected to be able to give information back to product so that they could go and build a better product. And what that meant was we had one core product that was constantly improving in ways that helped every single customer. I’d say that there are a couple of places in Jane Street where you can see maybe a similar model. I think Webs actually has a similar model where they have a Mantle, but I think that for the most part, as a desk dev, I spend a lot of my time thinking about my desk. I spend some time thinking about what things maybe could be contributed back to core products that can help all of Jane Street, but I’m not sure that necessarily feels like the crux of my job.
And I think that maybe that was the one thing that felt different structurally between the two in terms of my core responsibilities on the day-to-day. But I think that there are pros and cons to it as all things.
So just to go back, so you mentioned Webs and as a goofy bit of internal terminology, there’s a core Webs team.
Yes.
And then there’s more of a kind of client services sub team called Webs Mantle. Mantle is a joke about core, which I’m not sure I necessarily approve of, but people don’t run naming decisions by me.
Whimsy is important.
I’m a firm opponent of naming whimsy. I feel like you have to have a small whimsy budget. I feel like it’s totally good to have a mildly whimsical identity filled name for your system. But if you have a whimsical name for every little subsystem in your system, you’re just going too far, it’s confusing people. You just need small - It’s not going to be as funny in six months as it is right now. I feel like people maximize the wrong thing sometimes in their application of whimsy, but I digress. But anyway, yeah, so it’s a team that’s trying to reach out to understand the people who are using Webs and again, in part enabling stuff and in part figuring how to turn that around into, “Oh, actually what do we need to do to improve Webs for everyone?” So the interesting thing about your team is, we don’t have this kind of striation of there are these kinds of devs and that kind of devs.
And you guys both work with the people who you’re building software for, and you also build real deal systems for supporting those people. So when you say giving information upstream to the people who are providing the underlying product, it’s just actually a much messier, there’s a network of different people providing different pieces of software to whom you would supply information. So it’s like, if nothing else, it’s more complicated.
Yeah. Maybe an easy example is we had this thing that parsed chats and turns it into RFQs and it worked for sales for a really long time. It turns out not only sales gets chats and turns them into RFQs. There are some places in which our prop desks are doing the same thing.
Right, because it turns out having these relationships with customers doesn’t strictly fall on the official sales trading teams. Sometimes there are traders on the prop desk who do essentially the same kind of
Communication. Exactly. And I think that at this point we figured that out and we made our product more generalizable so that it can go and be used for other desks who are not just sales. And I think that’s great. But I think it actually took us a really long time to go and poke around the firm and realize, “Oh wait, I think I’ve solved a problem that a lot of people have and I should share the system that I’ve built so they don’t have to build the same one.”
Right. So in some sense, part of the time you’re a forward deployed engineer and part of the time you’re just a regular software engineer building a generalizable service and figuring out when you should be operating in one key and when you should be operating the other is not always clear.
Yeah, it’s definitely not.
Are there things that you feel like, oh, there’s useful ways in which Palantir approached it that we could import? What are the good ideas there?
I think it really was Palantir’s secret sauce, and I really do think that this is the thing of -
Yeah, it’s a thing they’re famously good at.
It’s famously good at.
It’s one of the reasons why I want to understand it.
I think that there are some devs who are great at this user thing and they’re great at this product thing. I’m not one of those devs. I think that I’m great at this user thing and I love being able to work on a team of people who are great at building this product thing. And I think that at Jane Street, I think that you’re expected to be able to wear a lot of hats. And I think that this means that we have a lot of really, really exceptionally talented generalist devs, but I think that this also means that sometimes it’s harder when maybe you are much, much stronger in one place to figure out how can I relay race this idea to someone who’s going to be able to take it and work it even stronger than I would be able to do it? And I think that on my team, we’ve started restructuring ourselves a little bit into devs who are really good at this product thing and devs that are really good at this desk dev thing and starting to try to figure out how do we make it so everyone is enjoying their jobs and learning how to do the whole gamut, but also is very aligned to their strengths.
And I think sometimes at Jane Street, I think sometimes I just wonder if our desk devs had an easier way to call back and say, “Hey, here’s these ideas that I think maybe apply to more than one person.” And we had some sort of way to say, oh yeah, no, I’m seeing that too. It’s like maybe we could turn and get some product that was working for the whole firm maybe a little bit faster. And I do say faster because I do think it does happen at Jane Street quite frequently where it’s like you’ll see a bunch of the same idea pop up on a bunch of desks at various different times or maybe the exact same time. And then all of a sudden a bunch of people turn around and they’re like, oh wait, I built that. I built that. I built that. And then you get to have an opportunity of -
You’re like the Spider-Man thing point at each other.
And then it’s like you get the opportunity to turn that into a product. And I think that does happen quite frequently, but I think it just happens a little bit slower than I saw it happening at Palantir because it does kind of require a bunch of different people to all have built the same thing and then someone to notice and then someone to organize turning it into one product that the firm can use.
I mean, Webs is a little bit, I mean it’s both an example and a counter example of what you’re saying. Webs was a thing that was built by the desk dev team on one particular desk. And then over time they’re like, actually, this is very general and we should apply it to lots of places. It’s not like a bunch of teams built the same thing. Really one team built it and then figured out it was more general and figured out to generalize it. I think the thing that slowed that down was mostly just opportunity cost. It’s very valuable where it was and at first it made sense to work on it mostly in that area. But I think we did eventually work around to it, maybe more slowly than we might have otherwise. So that’s kind of in the direction of what you’re saying. I feel like the other thing I’m hearing from what you’re saying is maybe that we should specialize more at a human level.
One thing that’s really different at Palantir is they really hire different kinds of people for different roles. They should be, oh, this person would be a good FDE and this person would be a good core engineer and they point them in different directions. And we tend to hire people who we think are strong generalists and then we figure out where they go.
Yeah. It’s a very different way of thinking about people. I think it was a huge update for me when I joined Jane Street and especially when I started hiring at Jane Street to start thinking about people as like, okay, no, I’m hiring generalists, which was very different than the specialists I was hiring at Palantir.
So you said before that you’re kind of a recruiting snob.
I am.
And you liked a lot of what you saw of recruiting at Palantir, and then you came here. What has struck you now that you are pretty heavily involved in London and the hiring process there and also how that affects things globally, what strikes you as the good and the bad of Jane Street’s approach to hiring?
Jane Street hires very, very differently than we hired at Palantir, which I think at first I found really surprising. And I think that I was, I don’t know, I think at some point you start thinking that you just know things and then you come to Jane Street and I was like, wow, we hire entirely differently here, but the people here are very, very talented, so they must be doing something right. And it was trying to figure out, okay, you can use an entirely different process to still get very talented people. I don’t know. I found that really interesting.
I feel like I’ve in general been surprised over time of lots of things about how Jane Street works. You think, well, we’re just doing it in the natural correct way.
Oh my gosh.
And then you look out and other companies do very different things and are often very successful and you’re like, maybe there isn’t a natural correct way. Maybe there are many different correct ways of doing things and you should just keep your eyes open and learn from what other people are doing.
Yeah. And I think one of those things I think with Jane Street is I’d say that at Palantir when we were hiring, we were trying really hard to get somewhat confident that someone was really good at at least one thing. I think at Jane Street, we’re trying to get very confident that someone is good at software development, I guess, as a broad thing. So I think at Jane Street, I’d say that we tend to ask very similar questions several times in order to give a candidate maybe the opportunity to like, oh, maybe they did a little poorly on their first question of the day, but their second and third question of the day they came back and now I’m actually very confident that they’re very good at this thing that I’ve kind of tested many, many times. Versus at Palantir, I think we gave very different questions that were very hyper-focused on one specific, what we would call facet of a person.
And what we were really looking for was someone who had one thing that they were excellent at and then were good enough at the rest of things. And I think that -
So multiple different dimensions in which you’re looking and each read in those dimensions is lower confidence because you have fewer instruments applied, fewer readings of that property.
Exactly. In fact, you only have one. So it’s just you get one interview where you’re trying to read that property. And maybe that some of the interviews have little properties that overlap, but it was very much in a way it was like you’re putting one interviewer in the room. It’s very different than Jane Street. You’re putting one interviewer in the room in order to get one read and then you’re working off of that one read as saying, okay, maybe I’m not super confident in this thing because it was one interview, one interviewer, but we had a lot of faith in our interviewers in those moments. Versus Jane Street, I think we think about it quite differently. It’s like we want to give someone the opportunity to succeed over a bunch of relatively similarly shaped interviews where you have two interviewers in each room. It’s like we’re getting very, very confident on a set of very specific reads, I’d say here.
You are trying to, in those conversations, learn about multiple different things at the same time. When I think about things that I want to learn about in one of our interviews, are they a good software engineer who knows how to write code? A thing they’ll often look for is when you look at somebody, look at the actual code that they write on the board and you ask them to think about it, you know how you think about you can tell if somebody writes a lot because there are callouses on particular parts of their hands. And I feel like there are marks on how someone talks and thinks about code that comes from have they thought hard and worked hard about how do you build careful, reliable software, the habits of mind that develop from that kind of work. That’s one kind of thing I look for. I feel like I’m also looking for people’s communication skills. Can they explain something to you? Can they hear the thing that you’re saying when you point them in the direction of an idea?
Do they stop and listen and understand? So you’re this kind of back and forth how comfortable it is to work with someone is something you’re looking for. And this is a little bit of a trading industry specific thing, but we care a lot about people’s understanding of their own levels of confidence, this kind of second order knowledge of not only do you know a lot of things, but are you appropriately calibrated? Do you understand how well you know? Are you wildly over or underconfident? Another kind of thing we look for. So I feel like we have a bunch of interviews where the questions poke at somewhat different aspects, but we’re actually looking for an ensemble of different facets. And at least at their best, interviewers are evaluating multiple of these at the same time.
For sure.
I do feel like early on we were much stupider about interviewing. I feel like early on we’d go into interviews and be like, “I know some hard questions. I’m going to ask some puzzles and we’ll hire the people who are really good at solving the puzzles.” And that wasn’t completely useless, but it did succeed in hiring some people who really did not work out. That is not a good general recipe of just hire the best puzzle solvers.
Yeah. It’s hard to come up with a hiring process because I think in a lot of ways, I think the thing that you’re trying to hire at the end of the day is people who are going to be successful at Jane Street. So in order to build a hiring process that does that, you first have to figure out what makes someone successful at Jane Street. And I think that some of these things are kind of obvious, I guess. Maybe it’s like you have to be a strong communicator sometimes. You have to know how to write code if you’re going to be a dev. They’re all -
And to be clear, we were basically just talking about the software engineering engineering process. So maybe the writing code is not a terrible thing to want at least. Although there are different mixes of skills there that could be good.
Yeah, there are different mixes of skills. I think some people have a very deep understanding of maybe their programming language. I’ve had conversations with people who are coding in a programming language I frankly have never seen before in an interview, and I have to ask a lot of very basic questions. And sometimes you’re talking to a dev who has a very in-depth knowledge of the language that is that they’re using. Sometimes you’re talking to someone who’s just using the language as a tool. I think we want both kinds of these people at Jane Street, but I think it is the different levels of understanding of the tool that it is that you’re using. And so I think that there is just this thing about trying to turn what makes someone successful at a job into a series of interviews that I think is just an exceptionally difficult problem.
I’m actually really curious about the facet thing. What is an example of a facet or two that they would use at Palantir and have a whole interview focused on that facet?
So for instance, all of our interns got a learning facet. And so what we would do is we’d give them a programming language that they had never seen before because it was made up. We would teach it to them and then we would ask them to solve a bunch of questions in it. And the whole goal was to just understand how did they learn? How did they approach learning a new language? How did they approach relying on their interviewer in that moment? And it’s like you get other information about how they structure problem solving, how they structure code, because they’re writing code. But the goal of this interview was just how teachable is this person? How steerable is this person? How much do I know about their ability to basically learn something very quickly on the fly?
Would you tell them that was the goal of that interview or is it just a thing they’re not supposed to know and -
Oh no, we would tell them.
Okay.
Yeah. It’s like, “Yeah, this is the learning interview and this is what we’re going to be doing in this interview.”
And then what’s another facet?
We had a re-engineering, I think is what we called it, interview. So that was specifically focusing on debugging skills. So we gave them a whole program written in Java, I want to say. And then we gave them a series of client bug reports and then we would help them with the Java because a lot of people didn’t know it. And then we would try to see, oh, how do they navigate a code base that already exists? Can they understand the structure of the code? How do they tackle debugging? We had an interview just for decomping a problem, so we would just give them a very, very large problem. If you had to redesign the whole New York subway system from scratch, how would you do it? Kind of vibe. Just hearing how they would decomp it almost felt more consultancy, but you would end up with some sort of computer vague shape of a program that maybe you would want to write at the end.
And all of these things, maybe you get a little bit of problem solving overlap, you definitely get communication overlap, but they were all focused on something very, very specific we were trying to learn.
It’s interesting because I feel like there are echoes in what you’re saying of how Jane Street’s process works in the sense that we actually do have this handful of questions that we use that are meant to be more design focused and questions that trying to get at people to read and debug code and various other pieces like that. But it sounds like the systematic approach is very different. It sounds for Palantir, it’s like when you’re constructing the interview, you’re like, okay, what facets are we going to cover? And you go off and cover those facets. Whereas here it’s more like we have a kind of standard interview process that we use across a wide variety of people that again, trying to get into this generalist set of skills. And then sometimes we’ll be like, oh, for this particular kind of person, we want to test some different things.
And so we’ll pull in different questions and try and lean things a little bit. But it’s more like you feel like there’s a relatively wide shape of a distribution of what you’re covering and your questions are shifted a little this way and shifted a little that way, but it’s almost like these are these delta functions that you’re trying to like, I’m going to focus on this facet, I’m going to focus on that facet. I feel like this is a closed-minded biased thing to say, but I feel like our thing sounds much better. So what’s good about the facet approach that I’m missing?
It hires you very different kinds of people. So you can imagine that at Palantir, you interview someone and they crushed their decomp interview. So they’re really good at decomping problems. And maybe they just did meh on the coding bits or meh on the learning and the rest of it was just kind of like, I don’t know, fine. We felt like we had enough signal on this one very specific facet about this one thing that this one person was excellent at. And the interview wasn’t clouded by the things that they were maybe just less good at because the interview’s very, very focused on this excellent thing that allows you to hire what I would call a very spiky person. So they’re a person who’s very, very good at one particular thing. At Palantir, we had a lot of spots for people who were very, very spiky.
And so it’s like you could be a kind of person who wasn’t a generalist and still do very, very well at Palantir. I think at Jane Street, we might miss this person. I think that all of our interviews are covering skill sets that are broad enough, but not quite as hyper-focused in on can this person do this one thing really, really well that I think that we might miss like, oh, actually this person maybe would make a great desk dev because they’re really good at talking to users and decomping problems. We might not notice that because it might get maybe muddled by the different signal that you’re getting in that interview. Maybe someone maybe even looks a little bit worse than they really are because our interview is kind of testing the thought process that isn’t quite in the same framework of the way that this person thinks.
I feel like the two concerns I’d have about the facet approach, one is have you really done a good job of picking the facets? I’ll say it seems really hard of picking facets that conditional on people doing well on that facet, there really is a good spot for them. So I think you have to get a lot of confidence in that decomposition of people’s skillset. And I guess the other thing is just the fact that you’re taking a single measurement of it opens you up to a lot of noise.
Oh yeah.
Both false positives and false negatives. Certainly someone can be very good at learning things, but do badly on the learning facet you throw at them.
For sure.
And then you just don’t get to measure that dimension.
Yeah. And that happens all the time. And so -
I mean this is the big dark matter of hiring is that processes tend to be conservative and that they tend to want to hire people who will be good fits. Although I mean it’s different by organizations. There are lots of organizations that hire more aggressively and then also fire more aggressively, which is a kind of necessary. If you need to keep a high level of people, then you really do have to fire a lot of people. And we’re not huge fans of firing lots of people, so we try and build… Part of the structure of the recruiting process is tries to build people who are quite likely to be successful. But yeah, do you feel like there were… How much was the noise thing a problem? I feel like it’s somewhat of a problem here. Certainly the correlation between how people did an interview and how they did say in their internship and then how they did in their internship and how they did as full-timers is not perfect.
Even though the internship, you have people there for three months, so you have a lot of time to learn. You think it would be perfect, but no, it’s really not.
No. I think it was pretty noisy. I think that we also did… I think first of all, we spent a bunch of time trying to figure out what our facets were. So I think that it’s like you’d spend years because a recruiting cycle is long. So it’s like when you want to change something in recruiting, you have to think about, okay, let’s say I changed something in my recruiting process today. I’m going to make my first hire in a month. That person’s going to start maybe in another three months. I’ll find out how they’re doing a year after that. And so then in a year and a half, I’ll know whether or not this decision I made was good. That is not the kind of timeline I like working with when I’m making decisions. But I think that we spent a lot of time at Palantir going through a bunch of different facets and testing them out and figuring out which ones were better than others. And I think that we also were happy with a little bit more noise. And so I think that we were kind of conservative about the people that we would take and we wanted all of our interviews to have more false negatives than false positives. And so we thought really hard about maybe getting harder by some proxy interview, but I think that it was noisy for sure. And I think that we were optimizing in a way for people who were going to be noisier even when they showed up. Because you’re optimizing for a person who maybe does have a very specific job that it is that they can fill at the time.
And so it just looked very different than I think Jane Street where we try to optimize for something that looks very different.
I think we do also have roles that are more specialized in various ways, sometimes subject matter specialization. I mentioned we’re hiring people for this formal methods team and we have a somewhat custom interview process. It hits a lot of the standard engineering questions because it’s primarily an engineering role, but there’s other things that we look at there. I think performance engineers are kind of an interesting corner. I feel like this is a place where I actually worry that we’re actively making mistakes. I mean, we do have a lot of performance minded people and hire a lot of performance minded people, but I think our overall process, one of the things we give people advice on doing is we’re like, “You’re here for an interview. We want you to write some code for this problem. Please focus on writing clear and simple and understandable code and don’t worry too much about the performance details of this code.” And if you are trying to hire someone who’s going to be a performance engineer, it’s like you can ask them to do that.
And also in their real job, it’s actually important for a good performance engineer to know when to bother to do the performance engineering work and when it doesn’t matter. So it’s not like that decision isn’t important, but I don’t know, they have intrusive thoughts about performance all the time and it’s hard for them to not think about it. And it’s also hard for them to believe that whatever you say, really what you care about is how good they are at thinking about performance because that is -
It’s the job.
It’s a large part of what the job is. And I worry that we sometimes think that people aren’t steerable or are not good fits or something because even though they are perfectly good fits, but they’re just a different kind of person who thinks about the world differently and our normal interview process might underrate them.
It’s hard to figure out what habits are habits that an experienced hire is showing in an interview because this has just been the way they’ve been coding their entire careers and they’ll be relatively easy to change what habits someone is showing in an interview that we actually want them to have when we get here, but we just don’t want them to have in this moment in this interview and what habits we’re having in an interview where this is actually going to be hard to change, which is going to mean they’re not going to be successful at Jane Street. I think that especially when you’re hiring experienced candidates, it’s like we hire experienced candidates because we want their experience, right? I think that there’s also a thing where it’s just harder to interview someone who has habits in the way that they do things and trying to understand, oh, am I over selecting or under selecting or what’s going on here?
It’s really hard.
Yeah, we hire a ton of experienced people. I think half-ish of the people we hire are out of the internship program and half of them tend to be experienced, very roughly, and it’s great. I feel like we’ve learned an enormous amount from having experienced people start, but I do think it is just easier to evaluate the kind of straight out of school people. They’re just a more uniform set of people with a more uniform set of experience. People in their career specialize and they bake in whatever the environment that they are in, and it’s just more of a challenge to evaluate them properly.
I think it’s even when you look at campus hiring, I think it’s one of the things I think about in London is the kinds of students we see in London are different than the kinds of students we see in the US. I think that there are all sorts of places where there are some assumptions baked into some sort of standardization that maybe isn’t necessarily there. I think when you’re thinking about hiring a lot, I think spending a bunch of time questioning those assumptions I think is a huge part of your job. But also, to your point, it’s just acknowledging that you can never be perfect at this thing. So trying to figure out where is that optimization that I want to be having in this process?
Yeah, I think in many ways, Jane Street is a very efficient hiring machine and is very good at getting a bunch of very strong people to come in the door. But as the organization grows and you want to keep on hiring more people, you actually always have to be doing better than you did before. I think in some ways, happily, there’s a lot of opportunity for doing better because I feel like what you’re always doing in these processes is you’re searching for your keys under the lamppost. You know this analogy of there’s a guy who lost his keys and he’s looking at night under this lamppost and someone says, “Oh, you need help looking for your keys?” He’s like, “Yeah, yeah, I can’t find them.” He’s like, “Well where’d you lose them?” He’s like, “Yeah, I’m not sure.” But why are you looking under the lamppost?
He’s like, “Well, here there’s light.” But you could look in the other places, but you wouldn’t find it. And so you’re optimizing the search for where you happen to be good at finding people. And I think in recruiting, there’s something similar where you build a process that’s good at getting great people. And I think if you get too arrogant about it, you think, “Oh, this is just the way to find good people.”
Right.
But there are actually lots of different ways of finding good people. And I think over time, if you’re in an organization that’s growing and wants to increase the number of great people it can get, you have to widen out how you search and be open to trying different approaches and interviewing people in different ways and sourcing people in different ways and hiring people with different kinds of backgrounds for different kinds of roles if you want to keep on expanding what your capabilities are.
Yeah. You have to be willing to experiment, even though those experiments are really hard because you’re not going to get that signal on whether or not they worked for a really long time.
For a really long time. But I’ve been here 24 years. I feel like four years of an experiment doesn’t seem so bad timescaling. I’m just very old. So taking all that into account, what do you think we’re probably doing wrong? What are things where it’s like, “Oh, here’s the thing we could be doing better.” What do you think are our most likely mistakes?
I think that when I very first started helping out with more time spending in Jane Street hiring a couple years ago, I think my biggest thing was it felt like we were interviewing experienced hires very similar to the way that we were interviewing campus hires, so interns and straight to full-time people. I think that experienced hires, they just look different. I think that in terms of that conversation of how do you start expanding the kinds of profiles and the kinds of people that you’re capturing in your process, you have to be willing to engage a little bit more critically with that question of what makes someone successful at Jane Street? I think that I worry that we are missing people that could be successful at Jane Street because our process isn’t quite capturing the nuance of what some differently shaped people could look like who would still be successful here.
I think that there’s a question of what do we want Jane Street to be tomorrow? I hope that we’re never the kind of firm where if you want to change teams, you have to go through an entire new interview process. But that’s a thing that happens in some places where you end up with these very, very specific hiring processes that are hiring exactly for what’s successful in this team and you can be a very specific person. And so then it’s really hard to change teams. But I do feel like there’s some in between versus -
The generality is a real value. One of the things internally that we get to do is we get to pivot as an organization.
Exactly.
Internally, things are very dynamic and entrepreneurial, and at times we decide it’s like, oh, this is more important than that, and we’re going to move a bunch of people around and change what we’re working on. It’s actually as a slight aside, it’s fairly common that we have restructuring of how teams are organized or who works where or moving people around. And at some point I realized I had to be very careful in talking about this with people from some other firms where there’s a reorg and they grip their desk and be very nervous. I’m like, “What did I say?” We reorganize things all the time. It’s fairly chill. Then I realized there’s a lot of places where reorgs are a bad thing. It’s like the way in which you get the job that you like taken away from you and you end up with a new job that you don’t like anymore and all sorts of terrible things.
But I actually think being in an environment where small scale reorganization is happening kind of continuously and all the time is kind of great. My job’s been really different over many years. I’ve worked in lots of different groups doing lots of different things. I think it’s true for a bunch of people. And just the shape of the trading desks and the tech groups and all of that has just morphed a lot over time. And we’ve done even at the pure level of the technology we choose, we have optimized in the direction of people’s flexibility. The fact that we have tried to keep the number of programming languages down is a real thing. I’m super glad we added Python, even though it’s like, I don’t know, it’s not my favorite language in all the ways, but the ecosystem for ML and stuff is incredibly valuable. It’s been super useful to have.
It’s been a lot of work to make Python well supported here.
Even just having two languages can reduce people’s flexibility some of, oh, they spent all their time here only working in one language and maybe they don’t know as well, or maybe they’re not psyched about working in the other language. And even there, we’re trying to move into a world where people more broadly know both so that they can do whatever is important and not pick the technology because they happen to know it, but pick the technology because it’s the right one for solving the particular problem. And I think that flexibility has been incredibly valuable. And at a human level, having people who are themselves strong generalists has been really good. And also we’re big enough that some amount of specialization is totally called for. We have hardware engineers, right?
Yeah.
Our hardware engineers are actually unusually good at software. It’s actually a weird thing that most hardware engineers in the world don’t think like software engineers at all, even though you’d think they’re very similar things, but they’re really very different in a lot of ways. And I think in many ways I feel like hardware engineering just seems harder and more intellectually challenging than software engineering. But it is surprising, a lot of hardware engineers are not comfortable writing recursive functions where I feel like writing a recursive function, that’s the easiest thing in the world. But that’s super not how they feel about it, and they do stuff that seems super hard to me. In the end, we end up hiring hardware engineers who are much more like software engineers. But they’re still in practice, it’s mostly a waste of their talent to have them work on pure software problems.
The right thing for them to be working on is hardware. So in that sense, they’re less fungible even if they have a lot of the skills that would let them do the other things.
Yeah. I think you see the same thing in our front end engineering pipeline at Jane Street as well. It’s like a lot of our front end engineers actually totally could go and be writing backend code if they wanted to, but it’s like they do have this specialization that is super useful for Jane Street and very, very valuable.
And there’s a weird thing in which the outside world has somehow decided that front end coding is kind of second class. And you get lots of kids out of school saying, “I don’t want to do front end stuff. I wouldn’t want to be pigeonholed to doing front end things. Those are the bad things.” And it’s so cool. Front end stuff is really interesting. It’s actually really challenging to figure out how to design performant and flexible UIs and building good abstractions in that context is actually really quite interesting, but it’s somehow weirdly low status for reasons I don’t quite understand.
I don’t get it. I also think especially as a person who’s very user motivated, I think that sometimes you go and talk to someone and they’re like, “All I want is for this button to be blue.” And I’m like, “I can make your day.” That is just so fun because there’s this thing about front end engineering, which is there is these iterations with users that you can just have so quickly.
But in some ways it’s easy and true to say that there are lots of great people that we miss. That’s obviously true, but what would you do about it? You have the keys to the kingdom, you can change our recruiting process however you want. What would you do?
If I had the keys to the kingdom and I could change our recruiting process however we wanted.
As opposed to the complicated process of consensus building that is actually there where no one has the keys to the kingdom.
There are no keys.
We talk about it a lot and eventually come to a conclusion. But no, you have been given the keys, what do you do?
If I was given the keys, I would come up with some interview questions that I thought were testing things that people were going to be good at. And then I’d give every single person, this would be so technically difficult, I already feel bad saying this. Every single person, especially experienced hires, are coming through one interview where I expect them to crush it. Because I think the thing that I want is to just give every single person the opportunity to show off what they’re really, really good at when they come on site at Jane Street. And so I think that would mean if someone has a bunch of user experience and maybe they’re the CTO of something in their last job, it’s like I would want to be throwing them some very high level question that feels kind of CTO-y at Jane Street. There are some problems that we tackle at Jane Street that I feel like I wish I could go and throw at people and be like, “This sounds similar to your job. Here’s a Jane Street problem. What would you do with it?” And give them the opportunity to shine that way. If they were very performance-minded, give them a very specific question that tackles this thing. I think that there’s not enough opportunity for people to show off their specializations. And I’m not saying we need to hire people based on their specializations, but I do want to see them when I’m making a decision as to whether or not someone should come to Jane Street. So if I could do anything, I think I would add that interview.
So an attempt to be more adaptive to people’s background essentially of try and suss out what’s the most likely thing that they’re especially good at and see if you can have a question that lets them ring that bell.
Yeah, I’d give one facet interview. I don’t think I’d go all the way back to Palantir style. I think that was too much. And I think Jane Street, I really like what we do with our interviewing processes here, but I think I would love to make sure that every candidate has at least one opportunity to show off something that I think they’re going to be good at based on their resume. And I think that would be the one change I would make to just try to capture those people that maybe are showing up as just fine across all of our interviews, but could actually be excellent when they get here.
Awesome. All right. That sounds like a fun place to stop it.
Yeah.
Thank you so much.
Yeah, thanks for having me.
You’ll find a complete transcript of the episode along with show notes and links at signalsandthreads.com.