CommunityNews

CommunityNews

Our Engineering Team Used Python's AST to Patch 100,000s of Lines of Code

Why should you care?

How do we easily and scalably patch 100,000s of lines of source code? Read about how we used a simple yet powerful data structure – Abstract Syntax Tree (AST) to create a system that from one single central point, maps source code dependencies and in-turn patches all dependencies.

Abstract

A software system is usually built with assumptions around how dependencies such as the underlying language system, frameworks, libraries etc. are written. Changes in these dependencies may have a ripple effect into the software system itself. For example, recently, the famous Python package pandas released its 1.0.0 version, which has deprecated and changed several functionalities that existed in its previous 0.25.x version. An organization may have many systems using 0.25.x version of pandas. Hence, upgrading it to 1.0.0 will require developers of every system to go through the pandas change documentation and patch their code accordingly.

Since we developers love to automate tedious tasks, it is natural for us to think of writing a patch script that will update the source code of all the systems according to the changes in new pandas version. A patch script could be parsing the source code and doing some kind of find+replace. But such a patch script will likely be unreliable and not comprehensive. For example, say the patch script needs to change the name of a function get to create wherever it is called in the code base. A simple find+replace will end up replacing the word “get” even if it was not a function call. Another example would be that find+replace will not be able to handle cases where code statements spill over to multiple lines. We need the patch script to parse the source code, while understanding the language constructs. In this article, we propose the use of Abstract Syntax Trees (AST) to write such patch scripts. And then later, we present how ASTs can be used to assess code quality…

Read in full here:

This thread was posted by one of our members via one of our news source trackers.

Where Next?

Popular Backend topics Top

Scorpil
I dabbled in Phoenix for a while now, but never really got my hands dirty with it right up until now. Apart from the whole framework bein...
New
New
First poster: bot
What’s Next for Teal, the typed dialect of Lua - FOSDEM 2021. This is my talk about the latest updates on the Teal programming language,...
New
First poster: dimitarvp
I’ve spent the last year building keyboards, which has included writing firmware for a variety custom circuit boards. I initially wrote ...
New
First poster: malloryerik
Everyone outside of tech has heard of JavaScript, Java, Python, Ruby and even .Net, but few if any have heard of F#. However, F# may be o...
New
paulanthonywilson
Following up on the previous post on using UDP multicasting to broadcast and detect peers on a network, I create a registry of those peer...
New
First poster: bot
This post explains why Scala projects are difficult to maintain. Scala is a powerful programming language that can make certain small te...
New
tonyxrandall
When DoorDash approached the limits of what our Django-based monolithic codebase could support, we needed to design a new stack that woul...
New
brainlid
There is a new community resource available on writing “Safe Ecto Migrations”. When we get a migration wrong, it can lock up your product...
New
elbrujohalcon
Another week, another oldies-but-goldies post… This one about Test Driven Development.
New

Other popular topics Top

dasdom
No chair. I have a standing desk. This post was split into a dedicated thread from our thread about chairs :slight_smile:
New
AstonJ
poll poll Be sure to check out @Dusty’s article posted here: An Introduction to Alternative Keyboard Layouts It’s one of the best write-...
New
AstonJ
Inspired by this post from @Carter, which languages, frameworks or other tech or tools do you think is killing it right now? :upside_down...
New
AstonJ
I have seen the keycaps I want - they are due for a group-buy this week but won’t be delivered until October next year!!! :rofl: The Ser...
New
mafinar
Crystal recently reached version 1. I had been following it for awhile but never got to really learn it. Most languages I picked up out o...
New
Margaret
Hello everyone! This thread is to tell you about what authors from The Pragmatic Bookshelf are writing on Medium.
1143 25883 760
New
PragmaticBookshelf
Create efficient, elegant software tests in pytest, Python's most powerful testing framework. Brian Okken @brianokken Edited by Kat...
New
mafinar
This is going to be a long an frequently posted thread. While talking to a friend of mine who has taken data structure and algorithm cou...
New
New
First poster: joeb
The File System Access API with Origin Private File System. WebKit supports new API that makes it possible for web apps to create, open,...
New