YAZZZZ. I just raised my Python points over 9,000!
Sort of. I learned how to do something in pytest that blew my mind: creating fake attributes and methods of other Python objects. This meant getting better at understanding how modules and objects work. It was, mwah, so cool. But first, let's talk pytest and why-test.
Why test?
This was another culture shock about moving into tech: unit tests. In academia economica, you write a bunch of Stata code (.do files!), run a bunch of regressions, and hope it doesn't break and consider yourself replicable. But, ah ha, in tech world, your codebases are big, pieces of code interact in complicated ways, and it gets hard to hold it all in your head at once.
So you unit test!
The benefit of testing, as I see it, is that - as your codebase grows larger and more complex - it becomes harder and harder to know if any one change will break something else. The interactions and inter-dependencies get too complex. Often, multiple developers are working on different parts of the same, overall code. Unit tests, especially if they're small and discrete, can help. You break things down into small packets and make sure each joint in the pipe is working. They also force you to write small, discrete functions, and - by testing constantly - you can see if a small change on module A will lead to a break in module B, etc.
Then there's test-driven development (TDD), where the tests are written first and the code is written afterwards, constantly revised until the tests pass. I like this idea, but haven't really done it (yet!).
My current practice is to use tests as a way to get to know a codebase: I'll write new ones for code that isn't covered, and I'll tinker with old ones to see how they work.
A great video
I want to highlight this talk by Ned Batchelder at PyCon 2014, since it really informed the way I think about testing. The one thing that struck me that Ned said was, basically, treat your tests with the same respect and reverence as your code. Your tests don't need to be repetitive, non-DRY, ugly hunks of semi-identical tests. They can have objects. They can have good structure. pytest can help you write these tests in beautiful ways.
A great essay
This post by Kent Beck is also great at thinking through the how and why to test.
pytest (specifically, faking stuff in pytest)
Python has unittest built into the standard library (i.e. comes with Python), but I've been digging pytest. I reeeally like pytest. It has lots of very handy stuff either built-in or as easy pip installable extensions. I'll talk one the SUPER COOL thing I learned about: fixtures and monkeypatching!
One of the challenges of unit testing data-driven code (like a data science service, or ETL pipeline) is that it relies on external dependencies (e.g. a database, an API, a Kafka stream). To address this, we can use a monkeypatch fixture.
Fixtures (docs) allow you to pre-define baseline "states of the world" that you then test your modules against. They are related to the set-up/tear-down pattern where you set up, for example, a "fake" database object or a "fake" Kafka stream object, test your code with that, and then tear down the fake object. The convenient part of pytest’s fixtures is that setting up and tearing down is abstracted away from you. You just use the @pytest.fixture decorator in your test_module.py file, and pytest handles it for you.
Monkeypatching (docs) is one such fixture. It allows you to directly (and temporarily) modify existing modules, both from your own project as well as external libraries. For example, you could monkeypatch the requests library such that requests.get() always returns a specific string - but only in the context of your tests. So you don't have to worry about sullying your copy of requests, nor do you have to deal with the pain of setting it all up and tearing it all down.
Example: Fixing ye olde Odo bot
My beloved odo-bot was a Slackbot I built, long ago, at my old gig. odo-bot is early days Python for me, so forgive me the numerous obvious sins. If I re-wrote odo-bot now (DON'T TEMPT ME), I'd definitely include tests. For now, let's write a little unit test for one of his functions, Inspiration():
def Inspiration(event):
"""
A function to provide amusement and inspiration.
For now, pulls from Reddit's API and pulls a random top-rated post/image/URL.
"""
response = requests.get(f"http://www.reddit.com/r/{random.choice(SUBREDDITS)}/top.json?t=month")
if response.status_code != 200:
inspire = "I have no inspiration for you now."
else:
data = response.json()
data = data['data']['children'][0]['data']
inspire = "Try this: \n " + reddit + data['permalink']
# Er, re-writing this old function so that it's not directly calling the Slack API
# sc.api_call("chat.postMessage", channel=event['channel'], text=inspire, as_user=True)
return inspire
(source)
Ack! What a mess!
Anyway, the tldr on this function is that it takes an event - which is someone on Slack talking to Odo - and returns some "inspiring" Reddit post. What in the Lord? I don't even remember why I wrote this. What SUBREDDITS? Why are they global vars? Why did I think to put sc.api_call() in scope?! Oof.
Well, anyway. The core of this function is that it's calling the Reddit API (request.get()), parsing the response (if response.status_code, response.json()), and then posting it to the Slack API (sc.api_call()). The core of this function is parsing Reddit API responses correctly; that's what we want to test. But we may not want to test it by making actual API calls - this would waste our rate limits there. Similarly, we may not want to post test responses to our Slack API. Indeed, we want to isolate things to just testing the core functionality and removing any external dependencies.
pytest's monkeypatching can help us!
We see that response = requests.get() is the important bit. This returns a requests.Response object, which has different attributes and methods (docs). We want to somehow mimic the requests.Response.status_code as well as requests.Response.json(). However, while .status_code is a settable attribute, .json() works on requests.Response.text, which is not settable. Also, requests.get() will call the API; so if we monkeypatch the downstream stuff that's done to the Response() object (like .text, .json(), .status_code), we'll still be hitting the Reddit API - and bumping up against our rate limit. So let's monkeypatch .get() itself and return a FAKE RESPONSE OBJECT (aaaah).
We can do this from within our testing function, test_Inspiration(), itself. We temporarily modify requests to do what you want it to for the duration of the test:
def test_Inspiration(monkeypatch):
# The monkeypatch function is defined within the larger test of the function
def fake_get(api_url):
# OMG I know this looks bad, but I think it's okay...!
class Fake_Response(object):
def __init__(self):
self.status_code = 200 # our fake responses are always OK! yay
# some fake data, who cares, just gotta make
# sure the structure is the same as Reddit's actual API responses
self.text = '''
{"data": {
"children": [{
"data": {
"permalink": "some_reddit_link"
}
}]
}
}
'''
# Faking the requests.Response.json() method
def json():
return json.loads(self.text)
return Fake_Response
monkeypatch.setattr(requests, 'get', fake_get)
fake_event = {"channel": "nowhere"}
assert Inspiration(fake_event) == "Try this: \n http://www.reddit.com/r/some_reddit_link"
The way this works is:
- You define your testing function,
test_Inspiration(). Make sure to includemonkeypatchas one of your arguments. - Within that, you define your mock object or function,
Fake_Response(). This is the function that will temporarily replace the function/attribute/object you are monkeypatching (in our case,requests.get()). - Use
monkeypatch.setattr(obj, name, value, raising=True)to temporarily make the object's method (request.get()) return something else (Fake_Response(), instead ofrequests.Response()). Here's monkeypatch's source. - Proceed with the test, calling
Inspiration(fake_event), as usual.
What I think happens is that pytest will catch and parse monkeypatch when it looks for test_* functions in your test_*.py. Here's the official tutorial, similar to the above.