iptables-converter 92% tested
Reading the Python Testing Cookbook enlightend me, it was a big fun and had to be adapted to the real world example. Started to write tests for my iptables-converter in late feburary, yesterday it was finalized by achieving a coverage of 92% of the code. I assumed this value to be enough.
hans@jhx:~/gh/conv$ nosetests -v --with-coverage create a Filter group, f.e. filter ... ok check 3 valid policies, 1 exception ... ok 3 cases OK, 1 Exception ... ok flush filter group, 2 rules and an invalid chain ... ok create a new chain in filtergroup, ... ok insert a rule into an empty chain fails ... ok insert a rule into a non_existing chain fails ... ok insert a rule into a nonempty chain works at start ... ok append a rule to a chain ... ok try to remove a prefined chain ... ok try to remove a nonexisting chain ... ok try to remove an existing chain ... ok try an ilegal command ... ok create a Tables object, check chains ... ok nat PREROUTING entry ... ok mangle INPUT entry ... ok raw OUTPUT entry ... ok INPUT to not existing chain ... ok read non existing file ... ok read default file: reference-one, check chains ... ok procedure main ... ok Name Stmts Miss Cover Missing ------------------------------------- conv 177 15 92% 219, 228-246, 250-251 ---------------------------------------------------------------------- Ran 21 tests in 0.038s OK
To have reliable results, every method of the two classes is checked at least once, for an overall check a reference file is written:
hans@jhx:~/gh/conv$ nosetests -v --with-coverage iptables -F iptables -t nat -F iptables -N USER_CHAIN iptables -A INPUT -p tcp --dport 23 -j ACCEPT iptables -A USER_CHAIN -p icmp -j DROP iptables -P INPUT DROP iptables -t nat -A POSTROUTING -s 10.0.0.0/21 -p tcp --dport 80 -j SNAT --to-source 192.168.1.15 iptables -t nat -A PREROUTING -d 192.0.2.5/32 -p tcp --dport 443 -j DNAT --to-destination 10.0.0.5:1500
The iptables_converter.py reads this file and produces corresponding output after having reached the end of it. Comparing the output to the hardcoded reference within the testroutine should show no difference.
The produced output looks like:
hans@jhx:~/gh/conv$ python conv.py -s reference-one *raw :OUTPUT ACCEPT [0:0] :PREROUTING ACCEPT [0:0] COMMIT *nat :OUTPUT ACCEPT [0:0] :PREROUTING ACCEPT [0:0] :POSTROUTING ACCEPT [0:0] -A PREROUTING -d 192.0.2.5/32 -p tcp --dport 443 -j DNAT --to-destination 10.0.0.5:1500 -A POSTROUTING -s 10.0.0.0/21 -p tcp --dport 80 -j SNAT --to-source 192.168.1.15 COMMIT *mangle :FORWARD ACCEPT [0:0] :INPUT ACCEPT [0:0] :POSTROUTING ACCEPT [0:0] :PREROUTING ACCEPT [0:0] :OUTPUT ACCEPT [0:0] COMMIT *filter :FORWARD ACCEPT [0:0] :INPUT DROP [0:0] :USER_CHAIN - [0:0] :OUTPUT ACCEPT [0:0] -A INPUT -p tcp --dport 23 -j ACCEPT -A USER_CHAIN -p icmp -j DROP COMMIT
Writing these tests was funny. It's a good idea to understand what's going on. So lets dive into the tests a little bit:
#!/usr/bin/env python #encoding:utf8 from conv import Chains, Tables, main as haupt import unittest class Chains_Test(unittest.TestCase): '''some tests for class Chain''' def test_01_create_a_chain_object(self): """ create a Filter group, f.e. filter """ self.assertIsInstance(Chains("filter", \ ["INPUT", "FORWARD", "OUTPUT"]), Chains) self.assertEquals({}, Chains("filter", [])) filter = Chains("filter", ["INPUT", "FORWARD", "OUTPUT"]) self.assertEquals("filter", filter.name) self.assertEquals(['INPUT', 'FORWARD', 'OUTPUT'], filter.tables) self.assertEquals("-", filter.policy) self.assertEquals(0, filter.length) self.assertEquals( \ {'FORWARD': 'ACCEPT', 'INPUT': 'ACCEPT', 'OUTPUT': 'ACCEPT'}, \ filter.poli) ...
The first test tries to instanciate an object of class Chains(), it's name is filter and it shall have some prefined chains, i.e. ["INPUT", "FORWARD", "OUTPUT"]. The second assertion tests, if an empty dictionary equals to an instance of class Chains with no prefedined chains. Then an object is instanciated from class Chains with the prefined chains of a linuxkernel, then test, if it's name is filter an controls the names of the chains, their policy and their length. At last the dictionary is checked in the filter object, where the policies are kept.
... def test_02_prove_policies(self): """ check 3 valid policies, 1 exception """ filter = Chains("filter", ["INPUT", "FORWARD", "OUTPUT"]) filter.put_into_fgr("-P INPUT DROP") self.assertEquals( \ {'FORWARD': 'ACCEPT', 'INPUT': 'DROP', 'OUTPUT': 'ACCEPT'}, \ filter.poli) filter.put_into_fgr("-P FORWARD REJECT") self.assertEquals( \ {'FORWARD': 'REJECT', 'INPUT': 'DROP', 'OUTPUT': 'ACCEPT'}, \ filter.poli) filter.put_into_fgr("-P OUTPUT DROP") self.assertEquals( \ {'FORWARD': 'REJECT', 'INPUT': 'DROP', 'OUTPUT': 'DROP'}, \ filter.poli) self.assertRaises(ValueError, filter.put_into_fgr, "-P OUTPUT FAIL")
The second test proofs the possible policies, which are predfined by the kernel. An invalid policy is checked to raise an exception. This seems to be necessary to prevent the user from unexpected misbehavior of the converter.
... def test_03_tables_names(self): """ 3 cases OK, 1 Exception """ filter = Chains("filter", ["INPUT", "FORWARD", "OUTPUT"]) filter.put_into_fgr("-t filter -A INPUT -i sl0 -j ACCEPT") self.assertEquals(['-A INPUT -i sl0 -j ACCEPT '], filter.data["INPUT"]) filter = Chains("filter", ["INPUT", "FORWARD", "OUTPUT"]) filter.put_into_fgr("-t nat -A OUTPUT -j ACCEPT") self.assertEquals(['-A OUTPUT -j ACCEPT '], filter.data["OUTPUT"]) filter.put_into_fgr("-t nat -A FORWARD -j ACCEPT") self.assertEquals(['-A FORWARD -j ACCEPT '], filter.data["FORWARD"]) self.assertRaises(ValueError, filter.put_into_fgr, "-t na -A INPUT")
Here first append to the INPUT filter chain is checked, exactly the same is done on the OUTPUT and FORWARD filter chains. A try to do it on an invalid group na necessarily fails, the exception is expected.
...
Userdefined chains creation and deletion is tested, inserting rules into an empty chain also is expected to fail. To append a rule into a nonempty chain is checked also. Some more checks are done, even an illegal command is proofed to raise an exception.
And of course, the tables class is tested as well:
class Tables_Test(unittest.TestCase): ''' Tables: some first tests for the class ''' def test_01_create_a_tables_object(self): """ create a Tables object, check chains """ self.assertIsInstance(Tables(""), Tables) tables = Tables("") expect = {'filter': {'FORWARD': [], 'INPUT': [], 'OUTPUT': []}, \ 'raw': {'OUTPUT': [], 'PREROUTING': []}, \ 'mangle': {'FORWARD': [], 'INPUT': [], \ 'POSTROUTING': [], 'PREROUTING': [], 'OUTPUT': []}, \ 'nat': {'OUTPUT': [], 'PREROUTING': [], 'POSTROUTING': []}} self.assertEquals(expect, tables.data)
Are all the prefined chains build from scratch?
... def test_02_nat_prerouting(self): """ nat PREROUTING entry """ tables = Tables("") line = "iptables -t nat -A PREROUTING -s 10.0.0.0/21" line = line + " -p tcp --dport 80 -j SNAT --to-source 192.168.1.15" tables.put_into_tables(line) expect = ['-A PREROUTING -s 10.0.0.0/21 -p tcp --dport 80 -j SNAT --to-source 192.168.1.15 '] self.assertEquals(expect, tables.data["nat"]["PREROUTING"])
Is a long inputline translated correctly? Filter, nat, mangle and raw table checked also. Check exception on non existing chain raw INPUT.
... def test_07_reference_one(self): """ read default file: reference-one, check chains """ tables = Tables() expect = { \ 'filter': {'FORWARD': [], \ 'INPUT': ['-A INPUT -p tcp --dport 23 -j ACCEPT '], \ 'USER_CHAIN': ['-A USER_CHAIN -p icmp -j DROP '], \ 'OUTPUT': []}, \ 'raw': {'OUTPUT': [], 'PREROUTING': []}, \ 'mangle': {'FORWARD': [], 'INPUT': [], 'POSTROUTING': [], \ 'PREROUTING': [], 'OUTPUT': []}, \ 'nat': {'OUTPUT': [], \ 'POSTROUTING': ['-A POSTROUTING -s 10.0.0.0/21 -p tcp --dport 80 -j SNAT --to-source 192.168.1.15 '], 'PREROUTING': ['-A PREROUTING -d 192.0.2.5/32 -p tcp --dport 443 -j DNAT --to-destination 10.0.0.5:1500 ']}} self.maxDiff = None self.assertEquals(expect, tables.data) tables.table_printout()
This overall test checks reading of a file, and conversion to a result best known to be correct.
All these and some more tests are done in the hope to reach the goal of a stable and reliable piece of code. Recently a firend of mine said: Software without atomated tests is suspected to be broken by design. I agreed.