[00:00:00] [music]
[00:00:04] Welcome back to another episode of Tech
[00:00:06] Unhinch where tech gets human powered by
[00:00:08] code district. I’m your host Rabia
[00:00:10] Javeed and joining me today is Chared
[00:00:12] who is the CTO of core story. Over a
[00:00:14] 30-year career, he has held senior
[00:00:17] engineering and technology leadership
[00:00:18] roles at Crowd Bodics, Soilent,
[00:00:21] Platikica, Disney, and Sun Microsystems,
[00:00:23] scaling teams, rearchitecting platforms,
[00:00:26] and now leading transformation into a
[00:00:28] Genibi based product. Welcome to the
[00:00:30] show, Charat.
[00:00:30] >> Thank you, Rabia. Nice to meet you.
[00:00:32] >> Likewise. Well, Charat, we’ll start off,
[00:00:35] you know, with our initial questions.
[00:00:37] You’ve lived through web, SOA, social
[00:00:39] gaming, e-commerce, low code. Now, it’s
[00:00:42] the time of Gen AI. every one of those
[00:00:45] um was called a revolution back then.
[00:00:47] What pattern repeats every single time
[00:00:50] and what’s different about this one?
[00:00:52] >> Yeah. So um if you look at each of these
[00:00:55] revolutions, uh they’ve all changed
[00:00:58] computing and the world in some
[00:01:00] fundamental ways, right? So the web
[00:01:02] revolution basically made it where
[00:01:04] information was vis was available to a
[00:01:06] large population for example much larger
[00:01:09] than what used to uh have access to
[00:01:11] computing or uh to information. So made
[00:01:15] a big change because it actually was the
[00:01:17] first time that you could have computers
[00:01:19] talking to each other through fairly
[00:01:21] open protocols. you know there was there
[00:01:23] was SOAP and then now we have uh we have
[00:01:25] sort of rest but but the protocols
[00:01:27] changed and it changed the way computing
[00:01:29] was done where computers could sort of
[00:01:31] collaborate with each other the and each
[00:01:33] of them has changed the world in
[00:01:35] fundamental ways you know whether it was
[00:01:37] by access to information or whether it
[00:01:39] was in in new paradigms that the people
[00:01:41] used what’s different with AI right now
[00:01:44] is I think the the biggest thing is that
[00:01:46] AI touches judgment as well right so
[00:01:49] people are using AI for inference
[00:01:52] uh rather than just computing and there
[00:01:54] is a difference between that you know
[00:01:56] what we’re seeing is that generative AI
[00:01:58] is able to uh help generate more code
[00:02:01] it’s able to analyze large volumes of
[00:02:04] information and and generate reports for
[00:02:06] you stuff like that um it’s starting to
[00:02:09] basically touch judgment more than it
[00:02:12] touches just the just computing by
[00:02:14] itself I think that’s that’s the way to
[00:02:16] look at it and in a way it’s going to
[00:02:18] democratize even further I think access
[00:02:21] the information and access to knowledge
[00:02:23] and access to knowledge work that people
[00:02:24] have. So it’s it’s an interesting time.
[00:02:27] I’ll say that much.
[00:02:28] >> Char, you reoriented the whole product
[00:02:30] strategy around genai at core story. How
[00:02:34] do you tell the difference between
[00:02:35] something that’s actually going to
[00:02:37] change everything and something that’s
[00:02:39] just noise everyone’s trying to chase?
[00:02:41] >> It’s a good question. So to be clear, we
[00:02:43] didn’t actually reorient our product
[00:02:44] strategy around Genai. We were looking
[00:02:47] at um how to extract or we believed very
[00:02:50] strongly right from the beginning that
[00:02:51] specifications were very important for
[00:02:53] software development and we were looking
[00:02:55] at strategies to extract specifications
[00:02:57] from existing code bases um and also
[00:02:59] convert specifications into code right
[00:03:01] so we’re looking at both and we realized
[00:03:02] that genai was one of the technologies
[00:03:05] that would help us get there so that’s
[00:03:07] that’s really what we did right from the
[00:03:09] beginning the the biggest challenge for
[00:03:11] the problem set that we were trying to
[00:03:13] solve was the ability to to essentially
[00:03:16] understand rules and rule sets and and
[00:03:18] and sort of extract data from code
[00:03:21] bases, right? And extract meaning the
[00:03:23] semantics of a codebase. It’s not so
[00:03:25] much just the structure because
[00:03:26] structure is very easy. Plenty of static
[00:03:29] analysis tools that are available both
[00:03:31] open source and paid that can look at a
[00:03:33] codebase and extract structure from it.
[00:03:34] It can tell you which function calls
[00:03:36] what. It can tell you what the variables
[00:03:38] are and so on and so forth. what we were
[00:03:40] looking for was extracting semantics and
[00:03:42] that’s where inference is is important
[00:03:45] and that’s why we kind of started using
[00:03:47] genai right and again if you think about
[00:03:49] this whole thing around the judgment
[00:03:51] around you know what jai does for you um
[00:03:54] I think that that’s that’s really key
[00:03:56] right ji gives you the ability to infer
[00:04:00] stuff that you didn’t have before
[00:04:01] [clears throat] or which was really
[00:04:02] difficult before probably something that
[00:04:04] we would have done with huristics in the
[00:04:06] past or you know really uh complex X
[00:04:09] machine learning uh where we would have
[00:04:11] to train large pipelines to do this. I
[00:04:13] think it’s become a lot easier because
[00:04:15] these large language models whether
[00:04:18] closed source or open source or open way
[00:04:20] they’re all very similar in what they
[00:04:22] do. They they give you that ability that
[00:04:24] you never had before.
[00:04:25] >> We see that many companies have
[00:04:26] announced half cuts after rolling out AI
[00:04:29] in customer service back office
[00:04:31] operations and finance. How stable is
[00:04:33] that really? What do you think?
[00:04:34] >> It’s definitely a difficult time for
[00:04:36] many people in the industry. Right. So,
[00:04:38] so companies have announced huge job
[00:04:40] cuts in a long time. I think this is
[00:04:42] probably the first time that that these
[00:04:44] job cuts have affected sort of what you
[00:04:47] would consider to be um you know the the
[00:04:50] knowledge worker right and it’s pretty
[00:04:52] significant. So for those who are
[00:04:53] impacted it is hard and and I think for
[00:04:56] those who are not impacted there’s a lot
[00:04:58] of questions about how long am I going
[00:05:00] to be around right so there is that now
[00:05:02] that being said what I think we will
[00:05:04] will happen is that there is some
[00:05:06] initial euphoria almost that generative
[00:05:09] AI is going to replace all of these jobs
[00:05:11] but really what has happened is it
[00:05:14] hasn’t totally replaced jobs what it has
[00:05:17] replaced so it it may have replaced the
[00:05:19] physical act of doing something Right?
[00:05:22] So maybe it’s replaced the act of
[00:05:25] coding. What it hasn’t replaced is the
[00:05:27] act of engineering. And I make that
[00:05:29] distinction in some of my articles and
[00:05:31] stuff like that. And we could talk about
[00:05:32] it if we have time. So the point is that
[00:05:34] generative AI has replaced some of these
[00:05:37] acts but not the discipline itself. And
[00:05:39] I think what will end up happening is
[00:05:41] you will actually find that over time I
[00:05:43] suspect some of these jobs will come
[00:05:45] back or they will they will change the
[00:05:48] level of abstraction at which people are
[00:05:50] working. Right? That’s what I think is
[00:05:52] going to happen. Now, how stable is
[00:05:54] this? I don’t know. Uh what we’re really
[00:05:56] seeing, I think, or what I would I I
[00:05:59] think I am seeing is that generative AI
[00:06:01] is consolidating information and
[00:06:03] judgment in the hands of fewer people uh
[00:06:06] than we had before. And I think that’s
[00:06:08] actually going to come back to be a
[00:06:09] challenge for companies too, right? So,
[00:06:11] it kind of goes both ways. And I think
[00:06:14] the best way to look at it, it’s early
[00:06:16] days right now. So there’s a lot of
[00:06:17] euphoria that people are that companies
[00:06:20] I think are looking at and and you know
[00:06:23] we’re finding that even companies are
[00:06:24] walking back some of this right like if
[00:06:26] you look at companies like Microsoft
[00:06:27] sort of walking back their token
[00:06:29] matching that they did and so on and so
[00:06:31] forth so it varies but for those who are
[00:06:34] impacted I think the impact is genuine
[00:06:36] it impacts their livelihood it impacts
[00:06:38] their their daily sort of you know how
[00:06:40] they go about their lives and there’s
[00:06:41] nothing we can do about it for the
[00:06:43] immediate moment I think over time we
[00:06:45] wait for the expansion of the economy
[00:06:47] and we also wait for companies to sort
[00:06:50] of fine-tune where they think more
[00:06:52] humans are needed and where less humans
[00:06:54] are needed and I think that’s going to
[00:06:56] be where abstraction we’re all going to
[00:06:58] work at a higher level of abstraction
[00:07:00] than we have before. Well, that’s rather
[00:07:02] a um you know very challenging claim
[00:07:05] that you’ve just said and I hope that we
[00:07:06] can probably explore that a bit further
[00:07:08] because I believe that if AI is doing
[00:07:11] anything it is that it’s taking us away
[00:07:13] from the idea of abstraction and also
[00:07:16] you know it it’s leaving a large
[00:07:18] question on creativity as well and the
[00:07:21] jobs which are creative too perhaps so
[00:07:23] creative jobs you know they they’ve got
[00:07:26] this pool of subjectivity and
[00:07:27] abstraction to them and then that is
[00:07:29] supposed to be art perhaps. But then
[00:07:31] with AI being able to do everything with
[00:07:34] it, I am curious to learn more about
[00:07:37] that you know how do you define this
[00:07:39] abstraction which is probably you know
[00:07:41] going to built rather than you know
[00:07:43] lessened up. Moving to my next question
[00:07:45] which is also the extension of the
[00:07:46] previous question char when it comes to
[00:07:48] AI and engineering teams what are the
[00:07:50] most misleading assumptions if you can
[00:07:53] talk about three four of them that
[00:07:55] people or enterprises they walk in with
[00:07:58] versus you know what we see is actually
[00:08:00] happening
[00:08:00] >> you know this is this is the same thing
[00:08:02] that I said before I think there is only
[00:08:04] one misleading assumption and the
[00:08:05] assumption is that AI replaces the
[00:08:08] discipline but that’s not really true
[00:08:09] right it’s replacing a certain part of a
[00:08:12] discipline so if you look at software as
[00:08:15] or the creation of software as a
[00:08:17] discipline. What AI is really doing is
[00:08:19] it’s replacing the physical act of
[00:08:21] coding, right? It makes it very easy to
[00:08:23] generate huge volumes of code in a
[00:08:25] language that you may never have seen
[00:08:26] before. Even what it hasn’t replaced is
[00:08:29] all of the work that happens before you
[00:08:31] actually write a single line of code.
[00:08:33] So, it hasn’t been able to replace
[00:08:35] engineering by itself. on engineering
[00:08:37] when I when I mean engineering what I
[00:08:38] mean is that there’s always the act of
[00:08:41] understanding how things fit together
[00:08:44] what should fit together right what are
[00:08:46] the interfaces between different
[00:08:48] subsystems and this is true whether it’s
[00:08:50] software or whether you’re building a
[00:08:51] car it’s the same thing right that
[00:08:53] doesn’t change at all and AI hasn’t been
[00:08:56] able to replace that yes it’s able to
[00:08:58] you know it’s able to interact with you
[00:09:01] sometimes but it isn’t able to replace
[00:09:03] that part of the discipline right it
[00:09:04] isn’t able to replace that level of
[00:09:07] complex analysis yet. But the coding,
[00:09:10] yeah, sure, it can spit out huge number
[00:09:12] volume of code before you know it. And
[00:09:15] you know that that’s really what it’s
[00:09:17] doing. So that’s how I look at it at
[00:09:19] least, right? And I think that’s the
[00:09:20] most misleading assumption this this
[00:09:22] conflation that everyone is doing
[00:09:24] between engineering and coding. And and
[00:09:26] I think that’s that’s where the
[00:09:28] challenge is. You know, when you walk
[00:09:29] into a legacy system, what’s the first
[00:09:32] signal that tells you whether the
[00:09:34] problem is the technology or the org
[00:09:36] that built it?
[00:09:37] >> Not an easy question to answer, and I
[00:09:39] don’t think I want to speculate on
[00:09:40] whether it’s an org problem or a or a
[00:09:43] technology problem, right? We have
[00:09:45] customers who have mainframe systems
[00:09:47] that were literally built in the ‘ 70s,
[00:09:49] right? And they’re still running today.
[00:09:51] So, you can imagine, I mean, we’re
[00:09:52] talking about systems that were built
[00:09:54] 50, 40 years ago. And these systems have
[00:09:56] grown organically. And the real
[00:09:58] complexity comes from the fact that the
[00:10:01] business logic in those systems has
[00:10:03] basically changed and evolved over time
[00:10:05] as the needs of the business have
[00:10:07] evolved. The complexity comes from the
[00:10:09] fact that not all of this is documented.
[00:10:11] And also code isn’t always the best
[00:10:15] document for why a decision was made.
[00:10:18] Code tells you what decision was perhaps
[00:10:20] made, right? You you you know you
[00:10:22] decided to offer a discount maybe and
[00:10:25] and that’s fine. But what code doesn’t
[00:10:27] tell you is why that discount was
[00:10:29] offered. And I think that’s the
[00:10:30] complexity that most companies are
[00:10:32] dealing with, right? It’s not it’s not
[00:10:34] so much we have a broken tech org or
[00:10:37] anything like that. It’s just that you
[00:10:38] have an organization that has evolved
[00:10:40] over time and many of the people who may
[00:10:42] have made those decisions don’t exist or
[00:10:44] maybe they were not documented well
[00:10:45] enough. So over time, I think that’s
[00:10:47] that’s that’s the real challenge we see.
[00:10:49] So you know it’s it’s high time that we
[00:10:51] clear this distinction that you’ve also
[00:10:53] hinted upon in you know one of the
[00:10:55] previous answers that you said. People
[00:10:56] often confuse coding and engineering
[00:10:59] like you know they’re the same thing but
[00:11:01] they are not perhaps. So how
[00:11:02] [clears throat] would you explain the
[00:11:04] difference for our listeners in context
[00:11:06] of artificial intelligence?
[00:11:07] >> Yeah. So, as I’ as I’ve often said um or
[00:11:10] as I said earlier, the act of building
[00:11:14] something and this is true like I keep
[00:11:16] saying this is true whether you’re
[00:11:17] building a car, you’re building a you’re
[00:11:18] building software, you know, some of the
[00:11:20] some of the dimensions at least are
[00:11:21] true. The act of building something
[00:11:24] starts far before you actually build it.
[00:11:27] Right now, you can you can scrape
[00:11:29] together something like you you know you
[00:11:30] a car may be harder to do, software may
[00:11:32] be easier to do. Before you decide to
[00:11:34] build something, you’re you’re kind of
[00:11:36] thinking about how you want it to work,
[00:11:38] right? You have to decide about what are
[00:11:40] the pieces that fit together. What does
[00:11:42] it even look like? What’s the shape of
[00:11:44] this thing look like, right? I mean,
[00:11:45] take a car for example. What what does
[00:11:47] it look like? Is it going to be a sports
[00:11:49] car? Is it going to be an SUV? And then
[00:11:50] you’re starting to think about, okay,
[00:11:52] how what are the components that come
[00:11:54] together to build this thing, right? So,
[00:11:56] so you you start at a gross level. So,
[00:11:58] you know, at a very coarse level of
[00:11:59] granularity. So, you start to say, “Oh,
[00:12:01] yeah, you know, I I definitely need an
[00:12:03] engine. I definitely need four wheels. I
[00:12:05] need a steering wheel.” So on and so
[00:12:06] forth. Is it a four-seater car, a
[00:12:08] six-seater car, and so on. And then you
[00:12:10] start to kind of get closer and closer
[00:12:12] and more fine grained to this problem,
[00:12:14] and you actually are starting to think
[00:12:16] about, okay, what engine do I put into
[00:12:19] the car, right? What, you know, what’s
[00:12:21] the power of it? How many options do I
[00:12:24] offer? And then finally comes sort of
[00:12:26] the the spit and polish, right? Do I
[00:12:28] offer 10 different colors? Do I offer,
[00:12:31] you know, so on and so forth, right? Do
[00:12:33] I offer a sunroof? Do I offer this or
[00:12:35] that or the other? And each of these
[00:12:36] acts is actually a conscious set of
[00:12:40] decisions that you’re making and design
[00:12:41] that you’re doing and it all happens
[00:12:44] before anything even goes to the
[00:12:46] assembly line. It’s the same thing in
[00:12:48] software, right? The best built software
[00:12:51] tends to be software has to last long if
[00:12:54] it is truly enterprise software. Now you
[00:12:56] know we can we all will build software
[00:12:58] that is easy to use like you know
[00:13:00] everything from writing a macro in Excel
[00:13:03] is software or the creation of software
[00:13:05] and that has a one-time use or you know
[00:13:08] a two-time use for you the user but what
[00:13:10] it doesn’t have is not it’s not software
[00:13:12] that say your enterprise is going to
[00:13:13] use. So there’s a big difference there
[00:13:15] and there’s a distinction you have to
[00:13:16] make. But if you’re thinking about
[00:13:17] software that has to last, if you’re
[00:13:19] thinking about software that that runs
[00:13:21] an enterprise, I think these design
[00:13:23] choices become more important or more
[00:13:25] critical than the choice than sort of
[00:13:27] scraping together the code, right? Code
[00:13:29] is the last manifestation of this entire
[00:13:32] sequence of design that you’ve gone
[00:13:34] through. And at that point, the act of
[00:13:36] writing code is actually fairly
[00:13:38] mechanical, right? Even if you look at
[00:13:40] the art or the science or the art of
[00:13:43] software engineering over 30 or 40
[00:13:45] years, literally everything that people
[00:13:48] have talked about has been the same
[00:13:50] thing, right? Take something that is a
[00:13:52] complex problem that someone has
[00:13:54] specified to a user has expressed a need
[00:13:56] or a set of needs and then you gradually
[00:13:59] decompose that into smaller and smaller
[00:14:01] pieces until you basically can build the
[00:14:04] software for it. Right? whether we call
[00:14:06] it, you know, we had domain driven
[00:14:08] design, we have object-oriented design.
[00:14:10] Um, there’s just been so many paradigms
[00:14:12] around here, but they’re all trying to
[00:14:14] do the same thing. Take something that
[00:14:16] is huge, break it down into smaller and
[00:14:18] smaller components, figure out how these
[00:14:20] components fit together and then you
[00:14:23] express them in the form of code and the
[00:14:26] hope is that the code pretty much works
[00:14:28] well because now you’ve designed all
[00:14:29] these interfaces. Well, you know, that’s
[00:14:31] how I distinguish between the two. It
[00:14:33] almost doesn’t matter whether you’re
[00:14:35] looking at genai or not. It’s just
[00:14:37] that’s the act of building software.
[00:14:38] >> If writing prompts is becoming the new
[00:14:40] programming that we see it is perhaps
[00:14:43] you know where the engineers job is just
[00:14:44] to articulate what um should be built
[00:14:47] not build it. What does that team
[00:14:48] actually look like whether it’s working?
[00:14:50] That’s one. And two at what point does
[00:14:53] it start to feel like they’re building
[00:14:54] their own replacement?
[00:14:55] >> I’m not going to speculate on at what
[00:14:57] point it feels like someone’s building
[00:14:59] their own replacement. That’s a hard
[00:15:00] question to answer, I think. And that’s
[00:15:02] I don’t think that’s also the reason why
[00:15:04] people are doing genai based
[00:15:06] development, right? Like companies
[00:15:07] aren’t doing it exclusively to get to
[00:15:09] get rid of people. I think they’re doing
[00:15:10] it because we want to go faster. But
[00:15:13] let’s let’s think about the actual
[00:15:14] problem here. What Jai has allowed us to
[00:15:16] do is actually cast a very bright and
[00:15:19] shiny light maybe on the fact that there
[00:15:22] were effectively multiple stages in
[00:15:24] building software and that act of coding
[00:15:27] like I said it was probably the last
[00:15:29] stage right what it is doing is it is
[00:15:31] moving the level of abstraction up the
[00:15:33] to the left if you will or upwards if
[00:15:36] you will it depends on how we look at it
[00:15:37] but it’s basically moving things to an
[00:15:39] higher degree of abstraction when you’re
[00:15:41] not making decisions around when you’re
[00:15:44] not making decisions around how to or or
[00:15:46] or what text you’re putting into an
[00:15:49] editor, but the decision that you’re
[00:15:51] making is around how you want something
[00:15:53] to behave or how you want two subsystems
[00:15:56] to operate. That’s a higher level of
[00:15:57] abstraction and that’s just higher order
[00:15:59] thinking that we all we are all doing,
[00:16:01] right? It’s the same thing that that’s
[00:16:03] what’s happening today. And so the
[00:16:05] discipline of software coding is
[00:16:08] changing to where people are expressing
[00:16:10] in a more declarative fashion what they
[00:16:13] want the system to do but not expressing
[00:16:16] in very concrete terms how or what exact
[00:16:20] sort of statements that need to go into
[00:16:23] a compiler in order to generate the
[00:16:25] machine code to do it. Right? They’re
[00:16:26] kind of leaving that to the AI to sort
[00:16:29] of come up with. So that’s how I would
[00:16:31] look at it. It’s reducing the act of I
[00:16:34] mean it’s reducing everything down to an
[00:16:36] understanding of the system and really
[00:16:38] what is most important at this point is
[00:16:40] people understanding what system they’re
[00:16:42] building not so much what you know the
[00:16:44] syntax of the code it’s really
[00:16:45] understanding at a larger level okay
[00:16:47] what does the system do how is it going
[00:16:49] to do it and we’ve had different takes
[00:16:51] at this by the way over time right there
[00:16:53] they have been declarative languages in
[00:16:55] the past there are declarative systems
[00:16:56] where you kind of try and tell the
[00:16:58] instead of saying I want to add a I I
[00:17:01] mean I want to say I equals you know um
[00:17:04] J + K. You kind of say something like
[00:17:07] add I comma I mean J comma K or
[00:17:10] something like that. So there have been
[00:17:11] languages that do this just not very
[00:17:13] popular because again it’s a level of
[00:17:15] abstraction that people were
[00:17:16] uncomfortable with and now suddenly
[00:17:18] we’re all being asked to get very
[00:17:19] comfortable with this. It’s a bit of a
[00:17:21] it’s an interesting time.
[00:17:22] >> So a lot of gen AI automation is a thin
[00:17:24] wrapper on a model. What separates real
[00:17:27] intelligent automation from an
[00:17:28] impressive demo? If you start to think
[00:17:30] about the model as yet another piece of
[00:17:33] infrastructure around which you have a
[00:17:34] wrapper, then I think that’s fine,
[00:17:36] right? Like if you think about it,
[00:17:37] there’s a lot of software that’s been
[00:17:39] written in the past that were wrappers
[00:17:40] around libraries, right? We we used to
[00:17:43] use libraries all the time. It’s not
[00:17:44] like we’ve in the last I’m thinking like
[00:17:47] even in the early ’90s when I was doing
[00:17:48] some work on um realtime control
[00:17:51] systems, we used to use libraries like
[00:17:53] IMSL uh for numerical methods, right? No
[00:17:57] one ever handcoded integration or
[00:17:59] differentiation from scratch even though
[00:18:02] the numerical methods aren’t that
[00:18:03] complex to do it. And so we’ve used
[00:18:05] different things and we’ve always
[00:18:06] thought about software as assembling
[00:18:08] sort of things together. All right, it’s
[00:18:10] it so in a way that’s what it is and we
[00:18:13] we’re using an LLM today as underlying
[00:18:17] infrastructure to give us inference that
[00:18:19] we didn’t have capability of a few years
[00:18:21] ago. Now that being said, what
[00:18:23] distinguishes software that lasts from
[00:18:25] software that doesn’t? I think is is
[00:18:27] really that discipline of software
[00:18:29] engineering that I said before right
[00:18:31] when you build software in a way that in
[00:18:35] in a way that where the interfaces are
[00:18:37] well defined where the software actually
[00:18:39] comes together really well where the
[00:18:41] software is built with the so-called ETS
[00:18:44] in mind you know maintainability
[00:18:46] durability
[00:18:47] um performance and so on and so forth
[00:18:49] now that’s when software lasts right
[00:18:52] those fundamentals haven’t changed and
[00:18:54] they never will and that is again That’s
[00:18:56] that’s the distinction I keep making
[00:18:58] between engineering and and coding,
[00:19:00] right? [clears throat] It’s the same
[00:19:01] difference between I don’t know
[00:19:03] someone’s probably going to have a
[00:19:04] problem with it, but you know, maybe the
[00:19:06] difference between a BMW and a Ford
[00:19:08] Pinto, right? Like that. I mean, if you
[00:19:10] think about it that way, what’s the
[00:19:11] difference between the two? It’s not
[00:19:12] just the price. It’s not just the
[00:19:14] features that you have. It perhaps also
[00:19:16] is in the engineering of it.
[00:19:18] >> You have built an AI pipelines and the
[00:19:20] evaluations for them too. Why do so many
[00:19:23] teams skip the eval layer and what does
[00:19:26] that cost them later?
[00:19:27] >> Testing has always been a challenge in
[00:19:30] any software engineering endeavor and it
[00:19:32] continues to be the same. It’s it’s
[00:19:34] shocking that we can have so much genai
[00:19:36] and we still do the same thing, right?
[00:19:38] So an eval in a genai case is very
[00:19:40] similar to a test that you would have a
[00:19:42] unit test or maybe even an integration
[00:19:44] test that you would have in classic
[00:19:46] software development. People skip them
[00:19:48] because they’re just not sexy enough,
[00:19:50] right? people, you know, there’s so much
[00:19:52] pressure to build and ship something
[00:19:54] that people think about the eval as an
[00:19:56] afterthought. And one of the things that
[00:19:58] we do here at core story, we have
[00:20:00] consciously said that eval driven
[00:20:02] development is the way to go. So we have
[00:20:04] evals for our pipelines that we we
[00:20:06] basically will run. The whole idea here
[00:20:08] is that you know where you’re starting
[00:20:11] and where you’re ending up, right? And
[00:20:12] sometimes, I mean, we see this, you
[00:20:14] actually have worse results sometimes
[00:20:17] than you did when you started because
[00:20:18] geni is a little non-deterministic and
[00:20:20] all that. And the eval catches it and we
[00:20:22] can go back and fix it before it ships
[00:20:24] to production. So, so the short answer
[00:20:26] is they’re just not cool enough, right?
[00:20:28] They’re not you’re not they feel like
[00:20:30] you’re not delivering value to the
[00:20:33] customer. But I would argue that with
[00:20:35] the increased non-determinism of these
[00:20:38] systems or I shouldn’t say increased
[00:20:40] nondeterminism with the fact that these
[00:20:41] systems are inherently nondeterministic
[00:20:44] um I think it is fairly important for
[00:20:46] [clears throat] someone to know whether
[00:20:48] the system is behaving better or worse
[00:20:50] than it did before you started and you
[00:20:52] should be actually going to eval um and
[00:20:54] one of the things we do at core story is
[00:20:56] as part of what we deliver to customers
[00:20:58] we actually deliver an eval result as
[00:21:00] well so they know right at the get- go
[00:21:02] how well the system is performing.
[00:21:03] coming when they first see it and then
[00:21:05] we can tune it to get better over time.
[00:21:07] >> We see that some teams now review AI
[00:21:09] written code the same way they’d review
[00:21:11] code from any outside vendor before
[00:21:13] anything you know gets to the shipping
[00:21:16] part. Does that actually protect you
[00:21:17] your job or does the overhead cost more
[00:21:20] than the risk it’s preventing? A good
[00:21:22] question and I think a slightly
[00:21:24] controversial perhaps right so even even
[00:21:27] within the company there are there are
[00:21:28] there are multiple camps here around
[00:21:30] this and I think this is true of the
[00:21:32] entire industry the way we look at it or
[00:21:34] the way I look at it let me say this is
[00:21:36] not a course definition of anything I
[00:21:38] think the way I look at it is this I
[00:21:40] think it is important to have a certain
[00:21:42] degree of let me say mistrust but I
[00:21:45] don’t mean mistrust I actually mean a
[00:21:47] certain degree of um of discretion when
[00:21:50] you look at any code that’s generated by
[00:21:52] anyone. Code RA reviews exist for a
[00:21:54] reason. They exist for the reason that
[00:21:57] you know there are two things. One um
[00:21:59] they exist because it’s sort of a
[00:22:00] quality check, right? So we make sure
[00:22:03] that we haven’t made uh an an absolute
[00:22:06] blunder there or we haven’t made some
[00:22:07] kind of a mistake or we aren’t doing
[00:22:09] something that doesn’t make a lot of
[00:22:11] sense or potentially can come to become
[00:22:13] a problem for you later. They also exist
[00:22:15] in order to help get the whole team
[00:22:18] trained on different aspects of the
[00:22:20] codebase and that piece will go missing
[00:22:23] if you essentially have everything done
[00:22:26] with like genai code review. Okay. Now,
[00:22:28] so this is not about protecting jobs. I
[00:22:31] think this is about protecting quality
[00:22:33] and like I said a well-gineered product
[00:22:36] requires that level of quality in order
[00:22:38] for it to to sustain and be durable in a
[00:22:42] customer site, right? And it becomes
[00:22:43] even more complex when you’re thinking
[00:22:45] about something that is not SAS and
[00:22:47] something that’s maybe on tenant or on
[00:22:49] premise because those problems are
[00:22:51] entirely different. So I think a code
[00:22:52] review does help to catch it. Now code
[00:22:54] reviews are becoming more complex or
[00:22:57] more difficult because of the volume of
[00:22:59] code that geni can can actually create
[00:23:02] and this is true of anything that geni
[00:23:04] is creating. Right? So you know there
[00:23:06] there was a term that MIT recently co
[00:23:08] coined called cognitive surrender and
[00:23:10] and and the real idea here is that when
[00:23:13] there when you’re essentially flooded by
[00:23:17] so much data and so much information or
[00:23:20] so much text it’s really hard for you to
[00:23:22] understand what every piece of it is
[00:23:24] doing and you essentially end up saying
[00:23:26] oh it must be right. So you sort of
[00:23:28] surrender your cognition to the the
[00:23:30] output and you kind of say it’s okay
[00:23:32] right so I think it is becoming harder
[00:23:35] and so just like we have jai tools that
[00:23:37] generate code all of us are using
[00:23:39] combinations of jai tools to review code
[00:23:42] as well right so I can I can run I can
[00:23:45] run a jai based code review on my system
[00:23:48] to perhaps go over go over the codebase
[00:23:51] first and then I have a human sort of
[00:23:54] review it to say okay these may be
[00:23:56] trouble spots and Even even before the
[00:23:58] judgment didn’t change you know even
[00:24:01] before Genai as people were writing code
[00:24:04] the the amount of intensity or the
[00:24:06] intensity that you would apply to
[00:24:08] different code levels of code review
[00:24:10] were different right it’s
[00:24:10] [clears throat] something that’s a
[00:24:11] simple I’m just saying right maybe it
[00:24:13] was a 10 line change that wasn’t very
[00:24:15] core to something that you were doing
[00:24:17] sure you kind of reviewed it but you
[00:24:19] didn’t really you know you didn’t go in
[00:24:20] there with a fine tooth comb and try and
[00:24:22] find problems you’re doing the same
[00:24:23] thing today and it’s different for it’s
[00:24:26] just that the volume is larger and so
[00:24:28] you use tools to do a combination of it.
[00:24:30] So you maybe you have a tool that does
[00:24:32] the initial code review, you know, and
[00:24:34] and people have had these kinds of tools
[00:24:36] before, just not geni based. Uh you’ve
[00:24:38] had static analyzers that look at your
[00:24:40] code sometimes before they even get to
[00:24:42] you. You’ve always had llinters and and
[00:24:44] you know for to catch quality, not
[00:24:47] quality issues, but I would say these
[00:24:49] are textual quality issues in your code
[00:24:51] and stuff like that, right? Uh is the
[00:24:53] style matching what you have? But truth
[00:24:55] what happens you know um to your systems
[00:24:58] resilience when the same engineer you
[00:25:00] know who wipe coded the entire thing is
[00:25:02] now unable to you know review that code
[00:25:05] you know or debug it without AI
[00:25:07] >> that that will become difficult yes I
[00:25:09] agree um I think it’s going to become
[00:25:11] harder and harder as the volume of code
[00:25:14] goes in but again that underscores why I
[00:25:17] keep saying that strong engineering
[00:25:18] discipline is how this gets better not
[00:25:21] worse right let’s make the distinction
[00:25:23] between vibe coding and AI assisted
[00:25:26] software engineering which seems to be
[00:25:28] the new term that everyone’s adopting
[00:25:29] right all the cool kids are talking
[00:25:31] about that now and I think there is a
[00:25:32] distinction so when you are able to when
[00:25:35] you’ve designed your system in such a
[00:25:37] way that you’ve eliminated tight
[00:25:39] coupling between subsystems for example
[00:25:42] you now at least know that the behavior
[00:25:44] of one subsystem isn’t going to or or
[00:25:46] the code that’s written for one
[00:25:47] subsystem isn’t necessarily going to
[00:25:50] impact some other subsystem in this
[00:25:52] complex sort of piece that you’re
[00:25:54] building, right? So that’s how you do
[00:25:56] it. That’s the only way to do it. Go
[00:25:57] back to fundamentals actually.
[00:25:59] >> But you know, we see that some people
[00:26:01] they’ve already started using the term
[00:26:02] software engineering extension where the
[00:26:05] discipline gets quietly hollowed out and
[00:26:07] no one notices until it’s too late. Are
[00:26:10] we moving towards that time?
[00:26:11] >> I think that there’s a lot of hype in
[00:26:13] the market, right? So the hype cycle is
[00:26:15] at its fastest right now where
[00:26:17] everyone’s talking about things like
[00:26:18] this. I think there is some risk perhaps
[00:26:21] to it but I think that the risk will go
[00:26:23] away because over time what we’ll again
[00:26:26] realize is that systems that last are
[00:26:29] built that are are actually engineered
[00:26:32] really well and I think that’s the core
[00:26:33] of the thing right like I keep saying
[00:26:35] from the beginning the distinction
[00:26:37] between engineering and coding is I
[00:26:39] think what people need to think about
[00:26:41] not so much the fact that oh this whole
[00:26:44] thing just gets eliminated I don’t think
[00:26:45] it does and I think that I is part of
[00:26:48] the challenge Right. And I honestly I I
[00:26:51] hope that the incoming software
[00:26:54] engineers of today or tomorrow don’t
[00:26:57] assume that engineering is vibe coding,
[00:26:59] right? Like I hope that doesn’t happen.
[00:27:01] Uh because we really have to we we
[00:27:03] really need to be able to train the next
[00:27:05] generation and the next generation and
[00:27:06] the next generation and so on and so
[00:27:08] forth.
[00:27:08] >> Yeah, fingers crossed on that. So, you
[00:27:11] know, we’ve been we’ve spent this whole
[00:27:12] conversation on the traps. So if I flip
[00:27:14] this now, where has AI genuinely earned
[00:27:17] its keep in your work that you’re doing?
[00:27:19] >> There is no doubt that generative AI has
[00:27:22] helped with a lot of productivity gain,
[00:27:24] right? So things like the act of coding,
[00:27:27] right? I say this half jokingly, but I
[00:27:29] also say this with some degree of
[00:27:31] seriousness in that when I so so when I
[00:27:34] was a well I wasn’t a kid, maybe I was a
[00:27:36] teenager, right? I started coding when I
[00:27:38] was a teenager in like the early ’80s. I
[00:27:40] used to enjoy the act of coding because
[00:27:42] it was just the act of creating right I
[00:27:44] would I would basically look at I don’t
[00:27:46] know a problem something as simple as
[00:27:48] hey can I draw a circle on a screen I
[00:27:51] mean I’m just using that as an example
[00:27:52] and sure you know I I use that example
[00:27:54] because back in the day I so I used to
[00:27:57] program on a Sinclair spectrum in basic
[00:27:59] and spectrum basic actually had a circle
[00:28:02] command and so it would basically draw a
[00:28:03] circle on screen fairly easy but then
[00:28:05] what I realized is the circle was
[00:28:07] actually not like a real circle because
[00:28:09] it would get pixelated and I started
[00:28:11] looking at that to say hey how do I
[00:28:14] actually draw a circle right like what
[00:28:16] is the equation that defines a circle
[00:28:18] and and you know the equation is a
[00:28:19] trigonometric [clears throat] equation
[00:28:21] and so I basically built a circle
[00:28:23] subruine or is it yeah I think it’s
[00:28:25] called a sub routine in basic I don’t
[00:28:26] remember anymore a function something
[00:28:28] like that I built a a circle subruine
[00:28:30] that essentially used trigonometric
[00:28:32] functions to to draw that circle and it
[00:28:34] took like three times as long because
[00:28:36] you’d be calculating all of this right
[00:28:38] but the fact was it was fun to do that
[00:28:40] solve that little problem which feels
[00:28:43] insignificant when you think about it
[00:28:44] today but it was fun to do it and and it
[00:28:46] was more the act of almost creating
[00:28:49] something with an with a device that
[00:28:52] really couldn’t do much. Yeah. The weird
[00:28:54] thing is that and and then you know my
[00:28:56] career became my career and I kind of
[00:28:58] got in more and more into management and
[00:29:00] so on and so forth and I stopped writing
[00:29:01] a lot of code. I used to write it a
[00:29:03] little bit but not much. And we actually
[00:29:05] weirdly made the world more complex with
[00:29:07] the cloud and you know everything
[00:29:09] requires you to assemble like five
[00:29:11] different things before you can even get
[00:29:12] started and and the world became more
[00:29:14] complex. But the weird thing is off late
[00:29:16] I find myself actually writing a lot
[00:29:18] more code again. And and it’s actually
[00:29:20] become more fun. I’m not thinking about
[00:29:22] code in the sense that I used to, but I
[00:29:25] am actually thinking about the the fact
[00:29:27] that when I can when I [clears throat]
[00:29:29] have an idea and I can sort of put the
[00:29:31] pieces together in my mind, the
[00:29:33] engineering part of it and the design of
[00:29:35] what I want to build, it’s actually very
[00:29:37] easy for me to build the prototype and
[00:29:39] show it off to people. Right? That
[00:29:41] creative spark is actually back again.
[00:29:43] Not hasn’t it it kind of came back and
[00:29:46] it’s been a lot of fun to do it. I don’t
[00:29:48] think about a lot of the stuff that’s
[00:29:49] actually quite irritating when you’re
[00:29:51] doing software development like writing
[00:29:52] a make file, building a docker file, you
[00:29:55] know, figuring out how to, you know,
[00:29:56] wire up your package.json so that you do
[00:29:59] something like we don’t I don’t think
[00:30:00] about those things because the AI kind
[00:30:02] of does that for me and I don’t mind in
[00:30:04] the prototype even if it makes a mistake
[00:30:05] here or there. So there’s that. There’s
[00:30:07] also the true productivity gains of
[00:30:09] being able to consolidate information
[00:30:11] from huge volumes of from different
[00:30:13] systems. Yeah. So, you know, we I I have
[00:30:16] now a daily morning brief that I
[00:30:19] basically have one of these AI tools
[00:30:21] basically do for me. It it looks at my
[00:30:24] Slack, it looks at my email, it looks at
[00:30:26] documents that we’ve got, it looks at
[00:30:28] meeting uh records, and it basically
[00:30:30] says, “Hey, this is what, you know, your
[00:30:32] day looks like.” I think that’s that’s
[00:30:33] really cool because otherwise I’d have
[00:30:35] to spend maybe half an hour doing it. I
[00:30:37] still review it and I still spend some
[00:30:39] time sort of prepping for certain things
[00:30:41] mentally. But I think that’s a that’s a
[00:30:42] productivity game, right? There’s lots
[00:30:44] of things like that where AI I think
[00:30:46] will help. And I think the biggest thing
[00:30:48] that I’m finding and you know we I like
[00:30:50] I said there’s a distinction between
[00:30:52] engineering and and coding. But there
[00:30:54] are people in the company now who would
[00:30:56] have never sort of built a prototype or
[00:30:58] never sort of never built a prototype
[00:31:01] because they just didn’t know coding and
[00:31:02] didn’t want to learn a language or
[00:31:04] whatever. who are building prototypes,
[00:31:06] who are showing off demos, and it’s
[00:31:08] really amazing to see them doing that.
[00:31:09] You know, they’re able to go into
[00:31:12] customer discussions and while the
[00:31:14] customer is talking, build something
[00:31:16] like a quick prototype and show the
[00:31:17] customer and say, “Hey, is this what you
[00:31:19] were thinking about?” So, that kind of
[00:31:20] stuff I think is amazing. like the
[00:31:23] ability to the ability to create
[00:31:26] something like that where in the past it
[00:31:28] would have been you know a drawing or
[00:31:31] something like that is is I think really
[00:31:33] where these systems are going to help us
[00:31:35] right that to me is the big thing here.
[00:31:38] there is true gain there and the I think
[00:31:41] the the ability to create software the
[00:31:44] ability to create personal systems that
[00:31:48] um that you can use in your day-to-day
[00:31:50] life and and create quite complex
[00:31:52] personal systems I think is is the cool
[00:31:54] part right
[00:31:55] >> yeah know that was a great answer so um
[00:31:58] while we look forward what’s the
[00:32:00] assumption that most CTOs are making
[00:32:02] right now about AI ROI that’s going to
[00:32:06] look wrong in the hindsight side.
[00:32:08] >> I don’t know if I’m that much of a I
[00:32:10] don’t know if I should be the person
[00:32:12] giving people advice. Uh but I think
[00:32:15] practically there’s a couple of things
[00:32:17] and I don’t know about ROI by itself,
[00:32:19] right? Um but I will say this. I think
[00:32:21] everyone will be best served if they
[00:32:24] consider AI generated systems or AI
[00:32:27] outputs, they actually eval them before
[00:32:29] they put them into production. Right?
[00:32:31] That I think is going to be key. Don’t
[00:32:32] just assume blindly because I mean
[00:32:35] honestly you wouldn’t trust even your
[00:32:37] best engineer. You wouldn’t trust them
[00:32:39] to ship something into production
[00:32:40] without it being tested by your QA team.
[00:32:43] Why do you assume that AI is going to be
[00:32:44] any better? Right? Like you know we we
[00:32:47] know this. So so that’s one. And I think
[00:32:49] the second thing is that it’s probably
[00:32:51] more important for people who’ve been in
[00:32:53] industry a long time to work with the
[00:32:56] incoming batches of software coders,
[00:32:59] engineers, developers, whatever, and
[00:33:01] actually embibe in them the the
[00:33:03] discipline of software engineering. I
[00:33:05] think that is going to become more
[00:33:06] critical and we’re going to have to do
[00:33:08] it earlier in the cycle than later,
[00:33:11] right? like it’s no longer going to be
[00:33:14] where someone has to you just have to
[00:33:16] keep slugging code for 7 years before
[00:33:20] you’ be considered a senior and then
[00:33:22] someone will give you a design task. I
[00:33:24] think you start to have to do it early
[00:33:25] but then you don’t have that much
[00:33:27] experience and so I think it’s actually
[00:33:29] if I were a CTO if I were someone who’s
[00:33:32] been in the industry a long time I think
[00:33:35] it’s really the way you you pay it
[00:33:38] forward is by paying it this way. You
[00:33:40] actually help train these people. You
[00:33:42] actually groom them and you you kind of
[00:33:44] show them also the excitement of
[00:33:46] building software.
[00:33:47] >> Well, Char, you are a CTO yourself.
[00:33:49] [laughter]
[00:33:50] >> Yeah. So, I do that myself too, right?
[00:33:52] Like that’s the that’s the excitement
[00:33:53] that I have and I keep telling a lot of
[00:33:55] people the same thing. Um, which is, you
[00:33:58] know, you don’t what I tell people, you
[00:34:00] know, people come to me often getting
[00:34:02] into college and they want to know is
[00:34:03] computer science still a field that I
[00:34:06] should go into? Is it something that I
[00:34:08] should do? And I I think the the actual
[00:34:10] answer is yes. It’s even more important
[00:34:13] to go into the discipline of engineering
[00:34:15] than it is to sort of ignore it and walk
[00:34:17] away from it. Right? That’s what I would
[00:34:19] say. Just because we built you know CNC
[00:34:22] machines doesn’t mean that mechanical
[00:34:23] engineering went away. It actually
[00:34:25] became even more important to understand
[00:34:27] uh how to build these things. So, same
[00:34:29] thing.
[00:34:30] >> It’s now just, you know, a bit more
[00:34:31] challenging, you know, how to keep up
[00:34:33] with your own ability to be able to
[00:34:36] comprehend things, to be able to solve
[00:34:38] problems on your own, and you don’t have
[00:34:40] a sparring partner or, you know, a bot
[00:34:43] with you to help you. So, you know,
[00:34:45] being given that you’re a CTO yourself,
[00:34:47] if there’s simple test that you would
[00:34:49] have to give to other CTOs, a checklist
[00:34:51] perhaps for deciding whether to trust an
[00:34:53] AI generated output before it ships,
[00:34:55] what is on that checklist? Well, that
[00:34:57] checklist is very simple. It’s have you
[00:35:00] evaluated its outputs? Does it make
[00:35:02] sense? Can you trace the changes made in
[00:35:05] the system to a specification that
[00:35:07] actually works well? I think those would
[00:35:09] be on the checklist and and and recently
[00:35:11] I mean we we have a VP of engineering
[00:35:14] here at core story and he has started
[00:35:16] asking the team to do this which is can
[00:35:18] you write the summary or the synopsis of
[00:35:21] a pull request by by yourself humanly
[00:35:24] versus using AI to write it because
[00:35:26] that’s the first step in explaining what
[00:35:28] you just built right so if I think
[00:35:30] that’s that’s sort of been on my
[00:35:31] checklist I’ve always been that way in
[00:35:33] the sense when someone comes to me with
[00:35:35] something and says hey here look at what
[00:35:37] I did or whatever my First question is
[00:35:39] kind of, you know, let’s talk about can
[00:35:40] you explain what you did, right? Can you
[00:35:42] can you show me like I’m a kid. Uh can
[00:35:45] you can you explain it to me? Because
[00:35:47] the words we use to often abstract
[00:35:50] things sometimes cover a lack of
[00:35:52] understanding. Right? And so that’s what
[00:35:54] I would do is to say, okay, forget what
[00:35:56] AI said. Can you explain what it’s
[00:35:58] supposed to do for me? And I think that
[00:36:00] is part of the checklist. And and truly
[00:36:01] that’s where the whole concept of of
[00:36:04] sort of uh you know linking something to
[00:36:06] a spec comes in. That’s where eval come
[00:36:08] in, right? That’s what an a test does.
[00:36:10] It basically says this system should
[00:36:12] behave like this. Is it behaving like
[00:36:15] that? That’s and that would be on my
[00:36:17] checklist. Honestly,
[00:36:18] >> no, it’s this is a great insight that
[00:36:19] you just gave. Well, um moving towards,
[00:36:22] you know, our last question for this
[00:36:24] podcast. What’s the hardest lesson you
[00:36:26] um your work has taught you about
[00:36:28] technology or just life in general?
[00:36:30] >> I don’t know. Life in general, I there’s
[00:36:33] not much to say there. I mean life in
[00:36:34] general is you know it’s it’s it’s just
[00:36:37] life takes you on unexpected places and
[00:36:40] you know just enjoy the ride because
[00:36:42] it’s always fun. The destination is not
[00:36:45] always what you want right it may not be
[00:36:47] the destination you want but it’s still
[00:36:48] a lot of fun uh to get there. But in
[00:36:50] terms of technology by itself, I think
[00:36:52] the biggest thing that I have I I I will
[00:36:54] keep saying and I keep saying this over
[00:36:56] and you’ve probably heard me say this a
[00:36:58] thousand times in this in this 1 hour
[00:36:59] that we’ve talked or whatever, but speed
[00:37:02] without understanding is death, right?
[00:37:04] It’s not progress. And I think that’s
[00:37:06] the biggest thing like understand what
[00:37:08] you’re building, understand what you’re
[00:37:10] trying to build, understand what you’re
[00:37:12] building and then once you’ve built it,
[00:37:14] understand what you have built because
[00:37:15] the three are very different. Uh and I
[00:37:17] think that is where progress is going to
[00:37:20] come from. Not just in shipping code to
[00:37:23] you know saying I I built this. I don’t
[00:37:25] think that’s going to help at least for
[00:37:26] someone who is in the industry who is
[00:37:29] maybe uh young and starting up in
[00:37:31] industry. I think that’s the key like
[00:37:33] focus on the understanding not in a
[00:37:35] surface level sort of oh yeah you know I
[00:37:37] can write code kind of a thing. That’s
[00:37:39] that’s fun but [clears throat] it’s not
[00:37:40] what’s going to be a career.
[00:37:42] >> Yeah. Well thank you so much for this
[00:37:44] insightful and grounded conversation.
[00:37:46] >> Yeah. Thanks a lot, Rabia.
[00:37:49] [music]
