Our latest GCRC Interview is the latest in our series about startups and innovative companies in the legal market. The interest in legal tech and innovation is huge at the moment as VCs look to seize the moment amid the shifting legal climate.
The latest instalment of this series is with Noah Waisberg from Diligence Engine, a more established company than the other companies we have interviewed in this series.
To find out more about Dilgence Engine, you can have a look at their website: https://www.diligenceengine.com/
How much do you see in-house lawyers as being the drivers of the greater uptake of technology of the legal industry?
I think they’re huge drivers. They are both direct and indirect drivers.
Let’s start with indirect. In-house lawyers set the tone for law firms and one thing that we find when we go into law firms is that a lot of them realise that the clients would like more efficiently-created work product. They’re not coming out with that out of thin air, they’re coming out with that because of in-house lawyers. When we speak with in house lawyers, especially in medium-large corporates in competitive areas, we find that they are highly focused on costs and showing their superiors that they are doing something about costs. And them conveying this cost pressure to their outside counsel is important.
We have seen in-house lawyers be great direct drivers of efficiency technology as well. A number of serious laws firms have used our contract review software because their clients told them to. Sometimes firms go to clients suggesting they use our technology, but in a lot of the cases it goes the other way, with clients telling firms “we think this is something that can make an impact on costs we’d like you to try” – that’s more of the direct driver.
How much do you think it’s to do with pricing and costs per se or is it part of a wider move for more efficiency?
Many in-house lawyers are under pressure to reduce costs, so cost reduction is often a factor. That said, I think there is a push for greater efficiency. Nearly all in-house lawyers have practiced in large firms. They know there’s a lot of room for efficiency improvements in how these firms create work product. For example, most in-house M&A lawyers previously practiced in M&A at a law firm, and they’ve done the kind of due diligence contract review that our system can help with. They know that the traditional way of doing contract review is inefficient, highly time consuming and very prone to human error. And they often quickly grasp how our system can save them money as well as potentially get them better work product.
Just for our readers, can you give a description of DiligenceEngine and how in-house lawyers would use it?
Our software helps lawyers review contracts faster and more accurately.
Lawyers review contracts both for (i) drafting and negotiating as well as (ii) information extraction. Our system helps with information extraction—helping users determine what a company’s contracts actually say. This tends to come up in two areas: (a) transactional due diligence and (b) contract management database population.
Let me explain why contract review matters in transactions, using the example of an M&A deal. Say you were considering buying Jaguar. Jaguar would have a lot of contracts in place: supply, distribution, franchise, employee, financing, and many more. If you were going to buy Jaguar, you would be interested in knowing if Jaguar is what its owners say it is and what you would get if you bought it. There are essentially three questions packaged up in this. (1) What is the baseline—what are Jaguar’s current material agreements and what do they say? (2) Are there any liabilities that could come along with this purchase (e.g., potential lawsuits, environmental liabilities, agreements to indemnify others)? (3) Does anything happen to a company’s existing agreements if the transaction you’re considering happens (e.g., termination rights or notice requirements on change of control)?
The answers to most of these questions are in a company’s contracts. It takes a lot of work to look through the contracts and find what they say on issues you care about. For example, Jaguar could have a sales contract with a government department or rental car company that’s really very important to them, and that contract could require consent on change of control. The buyer needs to have their lawyers read through the target’s contracts to find information like this. It impacts how the lawyers do the deal, as well as even how much the target is worth.
Junior lawyers are typically assigned to do this contract review work. In the traditional approach to contract review, the junior lawyers would read through all the contracts, pull out deal-relevant information (e.g., consequences of change of control and assignment, term of agreements, governing law), and put the information into summary charts. Mid-level associates would then spot check the junior lawyer summaries for accuracy, and the summaries eventually would then make their way to more senior lawyers, who would figure out what to do with the information. Our software helps in this process. It quickly reads agreements for user specified provisions (change of control, term, exclusivity, governing law, and lots more), automatically creates summary charts, and gives users workflow tools to refine and understand results. Lawyers typically use our system to supplement their review teams. Instead of reviewing agreements from scratch, users can view them on our system highlighted with what the system has found, edit results, and track team progress. The system also helps coordinate review teams, including by allowing users to assign work and track when it’s completed.
Users who review contracts the traditional page-by-page way but using our system can get through their work in 20–30% less time. If they rely more heavily on our system, only spot checking results, they can get through in about 60% less time. The neat thing is that we’ve seen a user be more accurate as well as faster.
How much was Diligence Engine set up as a response to the arduousness of junior lawyers having to go through loads and loads of emails to try to find all the things that might be affected by M&A or any contract?
Large-scale email review tends to be in connection with litigation, and our software would typically be used in transactional or contract management work. People tend to be looking for different things when they do this work.
That said, our software definitely springs out of my prior work experience. Prior to DiligenceEngine, I was an M&A lawyer at a well-regarded New York City firm. I did contract review work and I supervised it. Though I think lawyers at my previous firm really did excellent work, I knew that there was a lot of room for improvement in how contract review work gets done at a lot of very good firms. Junior corporate lawyers can spend vast amounts of time doing contract review due diligence. They’re often quite smart and well trained, but contracts can be very complicated and they can be doing this work under heavy time pressure, and they sometimes make mistakes. Our software acts like a senior associate helping them along as they read, and can improve their speed and accuracy.
We often hear a lot about how automated technologies will take away the need of manual work of the kind of discovery work and kind of free up lawyers to do more lawyering, but what you’re saying is that this actually allows junior lawyers to still do that kind of work but at the same time with extra guidance and insight?
First off, there are big differences between contract review and discovery. The easiest way to illustrate this is by using costs as a proxy for detail. It costs about 60p per document to have a person review a document for discovery purposes. Large firm due diligence contract review often costs £100–500 per agreement. Contract review is significantly more complex work than discovery, and is not often outsourced (though outsourcing diligence is becoming more common). Our software enhances junior lawyers at contract review. It shows them potentially relevant provisions, and gives them tools to quickly improve what the machine generated. It takes some of the drudgery out of this work and helps juniors do work worthy of their billable rates. I would have loved having our software back in my junior and mid-level law firm associate days.
It’s using technology as an enhancement not a replacement because we hear a lot about lawyers being fearful about how technology is going to take away a lot of what lawyers do, leaving not very much left in what is already a difficult job market for younger lawyers. What do you think about automation as a threat to lawyers in a different way to what you guys are doing?
Automation is a threat to lawyers. First of all, I think our software and things like it just makes lawyer jobs better—few junior lawyers love diligence, and our system makes it less painful. But it also makes reviewers quicker: as I mentioned, we have found in doing speed and accuracy studies on our system, that someone can get through a contract in 20-60% less time using our software. So if you get through it in, say, 60% less time, that potentially means you need less lawyers. But only if you review the same amount of contracts, and that is not necessarily how it plays out. For example, in due diligence, what we see a lot of is companies requesting due diligence such that they would only like their lawyers to review the 100 most material agreements. A company can have 4,000, 40,000, or even a million contracts. It is likely not good value to review all of them at £200 a document, but it might be to review more of them if at a lower price. Our software can help lower the cost per document reviewed. We’ve seen lawyers use our system to review pools of agreements that would be uneconomical to review at standard rates.
I think it is uncertain whether legal technology destroys lawyer jobs. Assuming volume stays constant and you get more efficient, then technology takes away lawyer jobs. I just don’t know that it’s fair to assume that volume stays constant. I remember I was going out for drinks with a couple of people I used to work with – one of them was a partner practising for, say 25 years, one was a senior associate practising for, say, 10 years and the partner was talking about black line software. Its software pretty standard now but once upon a time lawyers had to prepare them by themselves, by hand, and she was saying how when that software came along, first of all it took them a really long time to get involved, and that was 20 years ago, but since they did become involved, the firms have grown dramatically. There are other reasons that firms have grown dramatically, but this technology made law firms more efficient. I think it’s more complex than just software makes lawyers faster at their work and therefore there will be less lawyers jobs. Maybe, but I don’t think so.
That’s a nice story to remind everyone that there’s always been technological innovations that have changed the legal landscape but the industry has always responded in its different ways. You mentioned Workshare. What is your competition and how do you assess DiligenceEngine in comparison to companies that do similar things to you?
There are two core differences between our system and the limited number of other companies in the contract review software market: accuracy and interface.
Contract provision extraction software accuracy is heavily tied to how a system actually works. Some contract metadata extraction providers use rule-based search technology to build their provision search models. We use machine learning technology instead, and there are significant differences between these approaches. Stepping back, in terms of finding contract provisions there’s two sort of broad ways that you could do it. One is rule based or keyword searches, like the Boolean searches you would use to search a library database. The other way is machine learning-based approach where the computer itself comes up with ways to improve the search. The advantage of rule-based search is that it is easy to build – you could get a basic rule-based extraction system built in days or weeks. The problem is that it won’t necessarily work well on unfamiliar agreements and poor quality scans. It is much harder to build a machine learning-based contract provision extraction system, but—once you get it right—it should be significantly more accurate on unfamiliar agreements and poor quality scans. This is critical, because a large portion of contract review work is on third party, unfamiliar contracts and poor quality scans. We’re the only company that publicises our accuracy, and this may say something. One of the reasons we publicise our accuracy is we think it’s pretty solid. The head of our research programme has a PhD in computer science from a well regarded computer science department.
As for our interface, we think it’s pretty great and that it combines well with our system’s automatic provision extraction outputs. Having done a lot of contract review work myself, I think our interface would be terrific on its own, even without our system’s automated results. There’s only so much I can say about our interface—it’s something you need to see for yourself.
If you’re considering contract review software, check out the alternatives. We think ours will be appealing.
One more question would be one we ask all of our interviewees. How much do you think that the current period of disruption in the legal market is actually a really excellent opportunity for innovation for lawyers?
This is potentially a very exciting time for lawyers. There is change, which means opportunity for easier advancement. As much as lawyers don’t like to be early adopters, lawyers that are early adopters seem to thrive. There are a lot of great examples of that with practice areas, where the leading firms in most practice areas are ones that started early. Giving a bunch of New York examples (since I know this market better: Skadden and Wachtell in M&A; Kirkland, Latham, Simpson Thatcher, and Weil in private equity; Weil in bankruptcy. There are lots of other good firms in these practice areas, but many of the ones I just mentioned seriously grew on the back of getting into these areas early. They took a risk and are still reaping rewards from it. The same thing can play out with taking a technological lead. I’m sure a lot of firms thought they could catch up quickly in M&A, PE, and bankruptcy, but few have. Lots of change in the market means lots of opportunity, but you have to take it.
Yes, it’s business isn’t it, things can go right or wrong
Exactly, that’s what I’m saying. I think it is a wonderful time for innovation in law. Depending on how dramatic change is, some firms may not make it through. But early adopters should thrive.