relaxdiego

This is part 3 of a series of articles. For other parts, see the introductory article.

Let’s Implement fetch()

First, let’s write the test:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
    @patch('ansible.modules.extras.cloud.somebodyscomputer.firstmod.open_url')
    def test__fetch__happy_path(self, open_url):
        # Setup
        url = "https://www.google.com"

        # mock the return value of open_url
        stream = open_url.return_value
        stream.read.return_value = "<html><head></head><body>Hello</body></html>"
        stream.getcode.return_value = 200
        open_url.return_value = stream

        # Exercise
        data = firstmod.fetch(url)

        # Verify
        self.assertEqual(stream.read.return_value, data)

        self.assertEqual(1, open_url.call_count)

        expected = call(url)
        self.assertEqual(expected, open_url.call_args)
  • Line 1 - We patch an Ansible function which we expect fetch() to call
  • Line 4 - The url that we will pass on to fetch(). We’re assigning it to a variable here because we want to verify if it gets passed on to open_url()
  • Lines 7 to 10 - We mock the IO object that open_url returns to fetch()
  • Lines 16 to 21 - We verify if fetch() returned the correct data and that it called the underlying open_url() correctly.

Run the test and see it fail. Let’s now write the code that makes it pass. First, add this near the top of your firstmod.py right after the first import line:

1
from ansible.module_utils.urls import open_url

Then, modify your fetch() method to look like this:

1
2
3
4
5
6
def fetch(url):
    try:
        stream = open_url(url)
        return stream.read()
    except URLError:
        raise FetchError("Data could not be fetched")

Notice that we’re catching a URLError here. Import that class as follows:

1
from urllib2 import URLError

Notice also that we’re raising a custom error class here called FetchError. This is so that we don’t have to write an except Exception catchall in save_data() which is poor error handling. So let’s add the following class to the file. I typically write this near the top, just after the imports.

1
2
class FetchError(Exception):
    pass

Run the test again and see it pass.

Let’s Implement write()

Add the following test to test_firstmod.py:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
    def test__write__happy_path(self):
        # Setup
        data = "Somedata here"
        dest = "/path/to/file.txt"

        # Exercise
        o_open = "ansible.modules.extras.cloud.somebodyscomputer.firstmod.open"
        m_open = mock_open()
        with patch(o_open, m_open, create=True):
            firstmod.write(data, dest)

        # Verify
        expected = call(dest, "w")
        self.assertEqual(expected, m_open.mock_calls[0])

        expected = call().write(data)
        self.assertEqual(expected, m_open.mock_calls[2])
  • Lines 7 to 9 - We expect the test subject to use the builtin open method when it writes the file. This is the de facto way of mocking it.
  • Line 13 to 14 - We check that the test subject opened the destination file for writing.
  • Line 16 to 17 - We check that the test subject actually wrote to the destination file.

You know the drill. Run to fail. Now write the code to pass it:

1
2
3
4
5
6
def write(data, dest):
    try:
        with open(dest, "w") as dest:
            dest.write(data)
    except IOError:
        raise WriteError("Data could not be written")

Like fetch(), this method also throws a custom exception. Add this to your code:

1
2
class WriteError(Exception):
    pass

Run the test again to see it pass.

Almost There!

Let’s run the linter against our module:

$ test/sanity/validate-modules/validate-modules <path to module dir>

That should list a few errors about documentation, examples, and the GPLv3 license header. Let’s fix that by modifying the top part to look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
#!/usr/bin/python
# This file is part of Ansible
#
# Ansible is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Ansible is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
# Make coding more python3-ish
from __future__ import (absolute_import, division)
__metaclass__ = type

from urllib2 import URLError

from ansible.module_utils.basic import AnsibleModule
from ansible.module_utils.urls import open_url


DOCUMENTATION = '''
---
module: firsmod
short_description: Downloads stuff from the interwebs
description:
    - Downloads stuff
    - Saves said stuff
version_added: "2.2"
options:
  url:
    description:
      - The location of the stuff to download
    required: false
    default: null
  dest:
    description:
      - Where to save the stuff
    required: false
    default: /tmp/firstmod
author:
    - "Your Name Here (@yourgithubusernamehere)"
'''

RETURN = '''
msg:
    description: Just returns a friendly message
    returned: always
    type: string
    sample: Hi there!
'''

EXAMPLES = '''
# Download then save to your home dir
- firstmod:
    url: https://www.relaxdiego.com
    dest: ~/relaxdiego.com.txt
'''

For an explanation of these strings, see Ansible’s Developing Modules Page.

Run the linter again. BOOM!

Test The Module Manually With Arguments This Time

We can write a playbook that uses our module now if we want to but I’ll leave you to do that on your own time. For now, let’s run it using the test-module script that comes with the ansible repo. Run the following:

$ <path to ansible repo>/hacking/test-module -m <path to first module dir>/firstmod.py -a "url=https://www.google.com dest=/tmp/ansibletest.txt"

Next, run the following to see what it downloaded:

$ cat /tmp/ansibletest.txt

Run the Sanity Tests

It’s a good idea to run the sanity tests every now and then so that you’ll know ahead of time if you’ve introduced some non-compliant code into the source. From your ansible repo run:

$ INSTALL_DEPS=1 TOXENV=py27 test/utils/shippable/sanity.sh

If the test fails, you may have introduced some erroneous code. Check the error messages and fix as needed. If you’re sure it’s not your fault, check if you’re working on top of an old upstream commit. If that’s that case, rebase your changes to the latest from upstream and try again.

BONUS: Get some code coverage!

Writing unit tests is great but blindly writing tests is not enough. So let’s see how much of your module’s code is covered. For this, we’ll use Ned Batchelder’s awesome coverage library. To get started, let’s install it:

$ pip install coverage

Next, we’ll use it with nose as follows:

$ nosetests -v --with-coverage --cover-erase --cover-html \
  --cover-package='ansible.modules.extras.cloud.somebodyscomputer' \
  --cover-html-dir=/tmp/coverage -w test/units/modules/extras/cloud/somebodyscomputer/

By running this single command, you’ll get two things. First is a quick test coverage summary via the terminal. Second is an HTML format of the same coverage report with information on which lines of your code has been executed by your tests. A green line means it’s been executed (and therefore tested) while red means it was not. Go ahead and open /tmp/coverage/index.html in your browser and be enlightened!

IMPORTANT: A fully covered module doesn’t automatically mean it’s bug free. But the coverage report is a great way to find out which parts of your code needs some testing TLC.

Holy Smokes!

I hope you’re as pumped as I am for getting this far. You do realize that, in just 3 articles, you went from zero to writing a fully tested Ansible module. That’s quite an achivement so grab a beer (or whatever is your cup of…ummm…tea) and celebrate your awesomeness!

When you’re ready, head over to part 4 where we’ll learn how to submit our code upstream. Alternatively, you can go back to the the introduction if you want to jump ahead to other parts.