Beyond the Chain

Settlement and Funds with Jan Huwald, network security engineer at Nash

September 03, 2020 Jan Huwald Season 1 Episode 3
Beyond the Chain
Settlement and Funds with Jan Huwald, network security engineer at Nash
Show Notes Transcript

This week, host Chris Fenwick is joined by Nash co-founder Thomas Saunders and one of our backend engineers, Jan Huwald, who are part of the Settlement and Funds team. They discuss Nash Pay and other protocols, state channels and a solution Nash decided not to move forward with.

Chris: Welcome to Beyond the Chain,the podcast from Nash that takes a deep look into blockchain technology and connects the dots of decentralized finance. I'm your host, Chris Fenwick and each week I'm joined by one of the founders of Nash, as well as another guest. This week, I'm here with co-founder Thomas Saunders.

Thomas: Hello!

Chris: With us as well is our backend engineer, Jan Huwald 

Jan: Hey, Chris.

Chris: Hi guys. So Tom and Jan work together on the Settlement and Funds team. Tom's probably familiar to many people from his work within the Neo community, but, perhaps Tom would like to introduce himself again.

Thomas: Sure. uh, my full name is Thomas Saunders and I'm from Minneapolis.  I studied philosophy in school and I did not end up wanting to be a roaming philosopher or teaching philosophy. One big part of philosophy is studying logic and metalogic so out of college, I took up computer programming and had been doing that for about 15 years now. In around 2016 - 2017 I started having interest in the Neo blockchain and there wasn't very much there for English users or any kind of developers so me along with the other founders here, and some other people ended up founding City of Zion, which is an open source development community centered on the Neo blockchain. With that we developed a lot of open source software for interacting with Neo that's still in use today. My main project in COZ was Neo Python and that was a Python implementation of the Neo blockchain. And that's still running today although it's taken over by another COZ person. The other thing I worked on was a Python compiler for smart contracts to be used on the Neo blockchain. COZ was going well and then Fabio had a great idea to start Nash so I left my job at the university to join Nash and the rest is history.

Chris: I'm quite curious personally about how you came to learn programming, but do you mean where are you taking computing classes alongside your philosophy classes? I also have a background in philosophy and have spent time studying logic and so on, but there's not very much directly involved in that kind of study that teaches you how to write Javascript.

Thomas: I'm curious now, too. I think it was just fun to me. The way I learned was being a part of an open source community that was centered in Minneapolis. So there were a lot of really active people that were young and I'm willing to learn and willing to share what they knew. So if you're in a community that is like that, you're really lucky. Being friends with those people and colleagues with those people sitting by myself and tinkering with things which, uh, is a great way to learn. And then 15 years later, that's still the best way for me to learn.

Chris: Yeah, learn by doing rather than just learning theory. So I think we should bring Jan here. Jan, I don't think it's familiar to anybody, uh, outside Nash where he's very dear to us. So, could you let us know a bit about your background please, Jan?

Jan: Yeah, sure. Not being familiar with me is kind of one of the goals. I'm one of those tinfoil hat privacy advocates, that started out kind of early. So, I had my 25 year programming anniversary just a short while ago and stumbled into network programming quite early on and that while still being a child. I was seeing a lot of programs using the master slave metaphor where one computer was completely idle being the slave and, answering to the master computer's demands I found that wrong that there should be a slave computer. I wanted all computers to be equal and that's how I got into peer to peer networks. I know it sounds silly, but that's how children's motivations go and yeah, most of those peer to peer networks were and are used for all sorts of illegitimate uses so yeah, obviously you want to not do that completely in the open. So I kind of transitioned from that or added to that diving into privacy technologies and playing around with VPNs and TOR and, I was always curious about what kind of information leakage you could have from just using the internet, which I was doing quite a lot early on. Both the peer to peer programming and my privacy focus kinda led me to co-found the Pirate Party and later on work on all sorts of cryptographic protocols. One of my pet projects is to, to install basically every peer to peer software that I can find and just play around with it and have a look at the protocol. One of those peer to peer softwares I was quite curious about was Bitcoin which seemed like a rather neat hack to bootstrap the funds distribution problem. So I was in the crypto money scene quite early and I actually also left quite early at the time when Bitcoin was around five euro or so. I didn't like the volatility and I didn't like checking how much of my assets are worth, several times a day. And I also didn't like the move towards a lot of centralization. So I'm really keen on the peer to peer aspect of Bitcoin and using a centralized exchange to do all your interaction with the blockchain sounded super silly to me, but that's, that was where the ecosystem was moving because the software just wasn't at a point where it was usable to normal people. Not sure if anybody knows PGP or any of the email cryptography software, I'm a big fan, but I also have to admit that the usability is close to zero for people who did not study computer science and did not have a cool open source community in their city to introduce them. 

Chris: I think to learn, to do some of these things, you have to go to a crypto party.

 Jan: Yeah, it depends on how you learn. I don't want to sound stupid. I'm a bit afraid of asking questions and rather spend hours reading manuals and trying out stuff by myself and that works going to crypto party also works. It's faster and probably much more fun but you basically need someone or something to hook you up initially it doesn't automatically come to you. And then I went into a PhD program and didn't think about cryptocurrencies a lot, except for ordering the occasional pizza but I always kept an eye on the new developments. I was super excited, for example, about Ethereum initially in the boat that historics and all the more researchy things. Um, and it was about two years ago when my interest was kind of rekindled for no particular reason. It's just like you haven't seen an acquaintance for a long time and you get the increasing urge to reconnect and in the same way I had that feeling. I should do something again in that space. In reading around and among other things came across and initially was a bit skeptical because when I read about it, it was before the ICO and it was built on Neo which I wasn't super familiar with. At which I'm always skeptical because they use like actual, non special purpose programming languages to write smart contracts and this sounds really complex to me and complexity is like the arch enemy of security. But after the ICO, I kind of convinced myself, okay this is a real thing. This is not a sham. I wanted to join. And apparently Nash wanted to have me. So that's how I'm here.

Chris: We're definitely very glad to have you here. How would you characterize what the settlements and funds team does? Is it essentially just dealing with how Nash interacts with the blockchain?

Jan: Yeah, that's one aspect of it, basically all the, all the protocol parts. So whenever somebody is not just using the exchange, not just placing an order, but once there is something that has an external effect that could be sending money to the exchange or withdrawing money or using our protocol to commit the state from the exchange to the blockchain, this is all settlement. The rest of the exchange is pretty well contained and we try to communicate as little as possible to the outside world because that also always brings complications. The settlement is whenever we do talk to the outside world, which is not just the blockchain, but also the other side of the equation, which is the user with whom we have to build the state ratchets.

Chris: Could you clarify how Nash works and the big picture, and maybe also say where the trade balances or the state channel contract balances are stored before they're written to the blockchain?

Thomas: So the big picture is let's say, for example, Ethereum, we have a smart contract on the Ethereum network that we wrote, and it has a couple of methods and one of them is deposit, withdrawal and then fill order, which I can explain a little bit later, but essentially when you want to move funds into your trading contract you're sending them from your account to, the smart contract. So, Nash never actually has control of your funds. They're sitting on the smart contract, and settlement and funds, if you were to boil it down to its most simple essence would be reconciling the differences between the orders that you see in the matching engine and what you see in the interface and what's happening on the smart contract itself. So you might deposit 10 Ethereum onto the smart contract and then trade it for a lot of USDC I don't know the number at the moment, but, then you want to withdraw the USDC. So there's a lot of different things happening there. The first is putting your Ethereum into custody of the smart contract, your next step would be actually placing the trade on our interface or with the API client that essentially tells our state channel, okay, this user now has zero Ethereum rather than 10 in the trading contract. And they have, let's say 20,000 USDC or something. So, the settlement and funds team makes sure that that update gets applied to the smart contract so that when you call withdraw on the smart contract that funds are taken from the smart contract and sent back to your address.

Chris: So the user makes maybe a bunch of trades between Ethereum and USDC and some other currencies. And then at a later date the result of those trades is written to the blockchain but where in the interim period is the result of the trades stored, like before it's written to the blockchain. 

Jan: But this is not part of the state channel. The state channel itself is a protocol, as a way to speak to each other, between the front end, this is the Nash website or the web app you're seeing and the exchange, which is the name for backend server. And this protocol, this exchange a few cryptographic tokens, which it could submit to the blockchain, but usually doesn't one of those tokens would be a place order token which basically says a particular user wants to place an order for one asset trading against another, with a given minimum and maximum rate and so and so many fees paid it max and then the exchange takes that order and puts it on the order book and trades and has a specific rate that this is filled with. We could then submit this cryptographic payload to the blockchain to commit this action to the blockchain, to actually do the trade. Unfortunately this is horrifically expensive so the blockchains are extremely compute constrained. So orders would have a really high overhead, if it would send every single order to the blockchain. This is why we have the state channel protocol to begin with. Instead of sending every single order, we do our totals from time to time together with the front end and say, okay, you did 200 orders. And in total, your Eth went down by 20, your Neo went up by 2,000 and then we sent this total to the blockchain and to do so, both the matching and doing and the client have to agree on those particular numbers and have to both sign them. This way we only sent one thing that is very cheap to compute to the blockchain engine, as opposed to a hundred that are very difficult to execute and consume a lot of fees. And this is what makes Nash possible.

Chris: So we're actually building technological solutions that are basically as trustless as we can make them and users can gain the absolute sort of maximum benefit of all that technology when they can run the APIs themselves on their own machine, but there are sort of certain, compromises people might use if they want the extra usability of our nicely designed front end client. 

Jan: Yes. This is basically one of the main differences between settlement and your average network engineering job. Usually you just assume the hackers are on the outside and you only try to defend against bad users. But we also have to assume that something on the inside is malicious and we have to write our code in a way that's safe, even if we are compromised.

Chris: Yeah. So as a kind of tinfoil hat, crypto person, you would say that you find Nash a trustworthy system, or at least certainly for people like you, who would prefer to just run their own code on their own machine to interact with us.

Jan: The important decision regarding usability is that your funds addressed are always safe. This is funds that are not currently in orders. So if they are an order we just have to make certain compromises and have to allow the matching engine to pick specific rates without submitting every minute detail to the blockchain. But at least from me who would never have big orders at least not for a long time, but with mostly a funds address this is basically my risk profile. I want to have funds somewhere where I know that under no circumstances, they can get lost only if I'm stupid and do a mistake, but not if somebody else gets hacked or tries to rip me off.

Chris: So we talked about state channels for the Nash exchange, but, uh, how important do you think state channels are for cryptocurrency payments in the future for accelerating transactions? And are there any major differences in terms of how they've worked from our implementations for trading?

Thomas: Yeah, so the Nash pay protocol would rest largely on kind of our state channel system that we already have for trading. So in the way that let's say the matching engine gets an update, that user has traded one Eth for 250 USDC and then sends that to settlement to reconcile with the chain. We would get a notification from a payment processor that user X has sent one to the payment processor and be able to credit that payment processors account with the USDC and then as that builds, the payment processor can reconcile to cash or Fiat or whatever cryptocurrency that they would like, but it all rests on the same interactions between the matching engine and the settlement system.

Chris: So that's for Nash Pay, but what about something like the Raiden network on Ethereum or Lightning network on Bitcoin are those simpler systems since they don't involve matching engine or is the matching engine basically just like a kind of manager for state channels, that are fundamentally the same?

Jan: Those networks I'd say are more complicated because they have more parties involved whereas our system has the central party that's not used for security properties, but for lightness properties, which in layman's terms means we do have our exchange that is used to make sure that the protocol always makes progress and never gets stuck, but isn't necessary to trust people to not lose their funds. And by having only one such party or protocol can be much simpler. So in your typical state channel network like Lightning, for example, you have a lot of involved parties and every single party could become malicious. So you need to have a relatively complicated network of insurances to attribute blame to specifically the one party that went rogue. We don't have that. And we have the opportunity that we know we are the good guys, so we can have an arbitrarily high penalty for us going rogue, because we won't do that. In the general Lightning network, you can't really have that because that would allow denial of service attacks against people to them.

Chris: Do you think that state channel systems will be necessary for cryptocurrencies to be accepted?

Thomas: Yeah, I think so based on the speed of blockchains, until we could have you know like a hundred thousand transactions per second on Bitcoin or Ethereum. There needs to be some kind of reconciliation layer that is capable of processing huge amounts of transactions, in an efficient way with the chain.

Jan: Yeah, I'd like to second that there are different names for slightly different proposals. So if you think about sharding, for example, this is an alternative to stay channels. But if you look at the implementation, then each shard, which is like a sub blockchain that executes early for some people, not for the entire mankind. Each of those shards has kind of a state channel protocol to talk back to the main blockchain and commit state. So the general principle of having a subset of the entire humanity, having state show among them and then only occasionally or on religious acts commit that to the main blockchain. I think this is going to stay.

Thomas: We did early on start looking at building our own chain that would solve this issue without state channels. And fortunately for us, we moved on to our current solution.

Chris: How would it deal with other assets? Would you have just like wrapped versions of every token?

Thomas: We didn't get very far in that, in those concepts we did carry forward. So one of the things that we wanted to think about was a trading specific DSL. So DSL is like a domain specific language. So solidity, which is the language that you write smart contracts in Ethereum is a DSL, which is kind of like a general purpose computing language for everything you need to do on Ethereum. One of our ideas was we wanted a general purpose trading computing language, where you would be able to declare certain trades when certain conditions are met and those would actually be executed within the chain. we haven't gotten there yet, but one of our ideas going forward is adding that script ability into our matching engine. So instead of placing a bunch of trades, you would upload a program that would be executed, you know, every, every heartbeat of the exchange. It's good to have lots of ideas and it's also good to recognize quickly when an idea is not a good one, but salvage any good seeds that came from an idea. So that was one of them.

Chris: I guess since you mentioned languages, maybe we could talk a bit about the languages you use at Nash. Most of our backend is written in Elixir, right? So could you maybe tell our listeners a bit about Elixir why it's an interesting language and why it's especially useful for this purpose? 

Jan: Yeah, so Elixir is a functional programming language, which makes it very hard to make certain classes of mistakes. Usually all of your state is unchangeable. You can only add new things, but you can't really change the old ones. This, especially in the blockchain context, makes it easy to model the data we get from a blockchain, which is also immutable and it makes it easy to not lose money by overwriting something. Elixir is built on top of a much older programming language called Erlang which originated somewhere in the eighties, interestingly, out of logics programming. So maybe that's why I like you Tom! Erlang is very good at concurrency, so you can run multiple things at the same time, very easily and distribute them across computers. And this is just a natural language for the typical cloud environment where lots of computers that talk to each other while one may occasionally fail, but you want the entire system to run despite the occasional local failure.

Thomas: Yeah. So I'd say Jan and the backend team work mostly in Elixir. Personally, I've always enjoyed working with Python, but haven't been doing that much. Been working a lot in TypeScript, which is the main language we're supporting in our API client and it's also the language that we use for our frontend, which is based on the open source React framework 

Chris: And what makes TypeScript such a good choice for the KPI clients and front end?

Thomas: I think at the moment it's because it shares the code base with our front end. So TypeScript is basically JavaScript if you've heard of that and JavaScript is kind of like a wild west programming language where you can do anything at any time and the interpreter will say, okay, it doesn't really matter. So TypeScript tries to impose some order on the, on the vagaries of, of JavaScript. So because we use a core, what's called our Nash protocol library, which is open source. We use the same library in that, for authentication, signing orders, signing transfers and everything on the front end as we do, we do in the API client. And that's mostly a matter of, of time restraints. At one point we still supported a Python API client but because of the divergent code bases, it made it hard to support that.

Chris: I mean, we have TypeScript for the front end, basically, just because it's like JavaScript and it's for the web.

Thomas: Yep. Yeah. So, but in the future we are currently working on a Rust version, which is a kind of a C++ language. We're working on writing our main protocol, which is currently in TypeScript in Rust. So when that's done, we'll be able to kind of have our main code base be in Rust for the front end and the API clients and, and,  , convert that to any language. then we'll be able to restore the Python API client, and then also supply a Java one and C-sharp. 

Chris: So within our system, at the moment, you can only deposit to our trading state channels from your Nash personal accounts. So we can sort of verify it's you going in and coming out. Some people seem to like the idea of being able to deposit to the state channels from external wallets. Is that something that would be possible?

Thomas: I would say our main constraint on that would be our KYC regulatory issues. 

Chris: Yeah. I mean, we would need to know which wallet the funds were coming from that were then being traded. Nash isn't necessarily just about creating an exchange. It's about creating a platform that deals with everybody's needs to make cryptocurrency accessible. I mean, the idea is that you store your funds on Nash in the first place, and then you just move it to the exchange, to trade and back again and not store in an external account and then move it to Nash personal accounts. And then to the trading contracts, I think people are too used to this UX pattern from centralized exchanges where you send it to the exchange and then you withdraw it again. And they need to sort of understand that with Nash you're not really storing it on the exchange and it's not really at risk when you send it to your Nash personal accounts cause this is just a wallet that you control. 

 Jan: To me, this is mostly just an optimization and sending less than less stuff to the blockchain, which also fits what we're currently doing. The transfers as they are right now were built to release as fast as possible while being secure. And that meant that and that to give and that something was optimizing all sorts of corner cases where we don't have to be so strict. This is something we're working on right now, so that transfers and sinking state co-operates more nicely so that you don't have to wait so long in most situations, because right now you wait so long because we treat every case as the worst possible case and do everything necessary to, to handle that. But most cases are much more simple and can be dealt with much more quickly. 

Chris: How does working at Nash differ from previous jobs you've had?

Thomas: I've never had the opportunity to work with so many super talented people. I've always been on good teams, but it's pretty amazing. Like Jan is incredibly smart, but also he's not show-offish about his intelligence. 

Jan: Yeah, I do enjoy the very diverse background of people at Nash, which just comes because people come from so many different places and employment histories. And I also enjoy the time zone distribution. Sometimes it's annoying that you can't directly talk to somebody, but with every person you have some overlap in the day.

And to me it often means that a problem that looked like a tough nut to crack in the evening was miraculously solved by the time I wake up in the morning and that happens so often with critical issues. And it's just a pleasure.

Chris: Are there any particular challenges of working at Nash think that it's more difficult than things you've encountered before?

Jan: For me challenges that the platform came together quite quickly. But that also meant that there's a lot of things around that you have to pay attention to that wouldn't be around if you take more time to build. This is the complaint every software engineer always has about too much craft around. There's always a way to do it more simple, but you usually only see it afterwards.

Thomas: I would say prioritizing what's most important for the product. What to work on on a day to day basis and on a month to month basis by kind of gathering what's important from our community and from our partners and trying to optimize the limited resources we have.

 Chris: Well, thanks for joining us today guys. Thanks, Tom. 

Thomas: Thanks, Chris and Jan 

Jan: Yeah, it was a pleasure. 

Chris: And Thank you for listening to Beyond the Chain. Subscribe with your favorite listening stream and you won't miss an episode. You can also find the full text of this episode on