JudoScript.COM Design principles of Judo the sport and the language
HomeJudo LanguageJuSP PlatformJamaica Language 
Judo ReferenceJuSP ReferenceWiki/WeblogTutorials/PresentationsDownloadsGoodiesFeedback  
Article: Java Software Unit Testing
 

Search

 

 By FreeFind.com











This article is old and is being consolidated into the book.
Please refer to the corresponding chapter(s) therein.
If the chapters or sections are not completed yet, you can use this article.
Refer to the examples as they are tested against the latest code.

Table Of Content

  1. Java Unit Testing with JUnit
  2. Problems with Writing Test Cases in Java
  3. Java Unit Testing with JudoScript
  4. Code Listings

Java Software Unit Testing

By James Jianbo Huang    February 2002       printer-friendly version

Abstract   Unit testing is critical to the quality of object-oriented software, because it solidifies the foundation -- the objects. This article explains the uses of a unit testing package for Java called JUnit, discusses the pros and cons of packages like such, and introduces how to do unit testing with JudoScript and explain why it is the way to go.


 

nit testing ensures the quality of independent logical units of the software. In the case of Java, such logical units are classes and sometimes the collaboration of a number of classes. Automated unit testing is essential to regression tests.

In this article, I will explain a Java unit testing package called JUnit, what it is and what it does. I will discuss issues regarding its approach, draw some clear-cut conclusions, and present unit testing in JudoScript to show that this is the way to go.

1. Java Unit Testing with JUnit

JUnit is a Java development package for writing Java code to test other Java software. In essence, it formalizes test cases and test suites in the form of Java classes, and provides a few assertion and reporting facilities, so that Java classes for test cases are developed consistently.

Since it is a testing tool, it can potentially be used by three types of engineers:

  1. Experienced Java developers
  2. Java-savvy QA engineers
  3. QA engineers with no Java programming experience

For experienced Java programmers, it probably takes less than an hour to start writing useful testing code with JUnit, because JUnit is just another Java package, and programmers use various Java packages day-in-day-out. If you do JUnit programming already, simply skip to the next section.

If you never program in Java, you are not up to using JUnit. Period. You can go to section 3 to learn how to use JudoScript for unit testing; otherwise, you will have to learn Java first.

This section is for Java-savvy QA engineers. First off, it is a good idea to print out the following image, the class diagram of JUnit's key package, junit.framework.

A class diagram depicts the relationships among classes. For instance, in the middle of this diagram lies the most important classes, junit.framework.TestCase and junit.framework.TestSuite and interface junit.framework.Test; both these classes implements interface Test; TestCase also extends class Assert (so that it can directly use all the methods in Assert). Since TestSuite only cares about Test instances, you can include other test suites in a (main) test suite.

To write a test case, you write a Java class that extends TestCase, in a couple of ways. Let me use a real test case to demonstrate. I need to test the following class, which contains a static method that runs a secure copy command to tranfer files between local and a remote machine running the SSH server:

public class SSHFactory
{
  public static void scp(String host,
                         String user,
                         String password,
                         String ciper,
                         String[] src,
                         String dest,
                         boolean recursive,
                         boolean toremote) throws Exception
  {
    ... ... ...
  }
}

The following is a run of a test case:

Listing 1. SCP Test Case 1
import junit.framework.*;
import com.judoscript.ext.SSHFactory;

public class MyTestCase1 extends TestCase
{
  protected void runTest() {
    try {
      SSHFactory.scp("ssh.judoscript.com", // host
                     "myuser",             // user name
                     "itspassword",        // password
                     null,
                     new String[]{ "temp/file1.txt", "temp/file2.bin" },
                     "temp/", // destination on the remote machine
                     false,   // not recursive
                     true);   // to remote machine
    } catch(Exception e) {
      fail(e.getMessage());
    }
  }

  public static void main(String[] args) {
    try {
      MyTestCase1 tcase = new MyTestCase1();
      TestResult tres = tcase.run();
      System.out.println("# of failures: " + tres.failureCount());
    } catch(Exception e) {
      e.printStackTrace();
    }
  }

} // end of class MyTestCase1.

For the test case, all you have to do is override the runTest() method for your testing code. Method runTest() is invoked by the JUnit framework to conduct the testing. This test copies two files under "temp/" in the current directory to the remote server. If it does not throw exceptions, it is considered successful. This criterion is rather loose, because even if it did not throw exception, we still don't know whether the files are copied correctly or not. One way to remedy this is to issue another scp() that copies these files back from the remote machine; if the identical files are retrieved, then we are confident that the file copies were indeed good. As I mentioned earlier, there are two ways to write test cases, so let us use the second approach.

Listing 2. SCP Test Case 2
import junit.framework.*;
import com.judoscript.ext.SSHFactory;
import java.io.*;

public class MyTestCase2 extends TestCase
{
  public MyTestCase2(String method) { super(method); }

  public void testSCP_NonRecursive() {
    try {
      // copy to remote machine
      SSHFactory.scp("ssh.judoscript.com", // host
                     "myuser",             // user name
                     "itspassword",        // password
                     null,
                     new String[]{ "temp/file1.txt", "temp/file2.bin" },
                     "temp/", // dest on the remote machine
                     false,   // not recursive
                     true);   // to remote machine

      // copy from remote machine
      SSHFactory.scp("ssh.judoscript.com", // host
                     "myuser",             // user name
                     "itspassword",        // password
                     null,
                     new String[] { "temp/back/" }, // local dest
                     "temp/file1.txt", // source on the remote machine
                     false,   // not recursive
                     false);  // from remote machine
      SSHFactory.scp("ssh.judoscript.com", // host
                     "myuser",             // user name
                     "itspassword",        // password
                     null,
                     new String[] { "temp/back/" }, // local dest
                     "temp/file2.bin", // source on the remote machine
                     false,   // not recursive
                     false);  // from remote machine

      // compare the files -- just compare their sizes
      long f1_len  = new File("temp/file1.txt").length();
      long f1b_len = new File("temp/back/file1.txt").length();
      long f2_len  = new File("temp/file2.bin").length();
      long f2b_len = new File("temp/back/file2.bin").length();
      if ((f1_len != f1b_len) || (f2_len != f2b_len))
        fail("SCP file copy failed.");

    } catch(Exception e) {
      fail(e.getMessage());
    }
  }

  public static void main(String[] args) {
    try {
      MyTestCase2 tcase = new MyTestCase2("testSCP_NonRecursive");
      TestResult tres = tcase.run();
      System.out.println("# of failures: " + tres.failureCount());
    } catch(Exception e) {
      e.printStackTrace();
    }
  }

} // end of class MyTestCase2.

In this case, we implemented a method called "testSCP_NonRecursive"; then, in main(), we passed this name to the constructor when creating a test case object tcase. This method has to be public. Why we need this approach?

In real world testing, many test cases require certain settings. For instance, if the remote host is not running SSH when we run the test case, it will fail but that is not a bug; it is a user error. Or the test is run on a different PC that does not contain "temp/file1.txt" -- another user error. In JUnit, you can set up and clean up these "test fixture" by overriding methods setUp() and tearDown(), which are called before and after the test by JUnit framework. Many times, these two methods actually take a lot more code than the test cases themselves. For example, in order to make the test case self-contained, we should prepare the files to be copied in setUp(), and remove them in tearDown(). In order to reuse setUp() and tearDown(), JUnit adopts the name-based test case methods as we saw in the second exmaple. This way, we can implement another test case called "testSCP_Recursive" in the same class to use the same set-up/tear-down methods.

To automate testing, JUnit provides class TestSuite. You can add multiple test cases and even other test suites into a test suite, and then run them altogether.

import junit.framework.*;
import com.judoscript.ext.SSHFactory;
import java.io.*;

public class MyTestCase3 extends TestCase
{
  public MyTestCase3(String method) { super(method); }

  public void testSCP_NonRecursive() {
    ... ... ...
  }

  public void testSCP_Recursive() {
    ... ... ...
  }

  protected void setUp() {
    // set up the test directories and files on the local machine
    ... ... ...
  }

  protected void tearDown() {
    // remove the test directories and files on the local machine
    ... ... ...
  }

  public static void main(String[] args) {
    try {
      TestSuite suite = new TestSuite();
      suite.addText(new MyTestCase3("testSCP_NonRecursive"));
      suite.addText(new MyTestCase3("testSCP_Recursive"));
      TestResult tres = suite.run();
      System.out.println("# of failures: " + tres.failureCount());
    } catch(Exception e) {
      e.printStackTrace();
    }
  }

} // end of class MyTestCase3.

This code is straightforward. The class itself implements two test cases. In main(), we created a test suite object and two test case objects, and added both to the suite. A single run of the suite does the job.

JUnit is a simple idea and simple framework. It tries to standardize the way people write test cases and suites for Java, in Java. In fact, in the article JUnit A Cook's Tour, it is clearly stated,

"the number one goal is to write a framework within which we have some glimmer of hope that developers will actually write tests."

Assuming developers are only willing to write Java code, JUnit cuts their excuses not writing any tests. The question is, is this a blessing for the QA team?

 

»»» Top «««

 

2. Problems with Writing Test Cases in Java

In a team environment, tests are run and maintained by QA engineers, not developers. Writing tests in Java has at least the following three problems. The root of the problems is, Java is a system language, designed for building robust software, not for doing things directly and quickly.

  1. Test cases in Java will take a lot of coding. Java has these characteristics:
    1. It requires a lot of house-keeping code -- code for class definition, method definition, exception handling ... that is not directly useful for the tasks at hand.
    2. Java APIs are detailed and low level, providing maximum flexibility. To do anything useful, however, you need many lines of code. Think of the SCP test case above, and try to create a directory tree with a few files in it for the recursive copy test.
    The consequences are:
    • Setting up necessary test environment may take a lot of coding and dependencies.
    • The more code, the more bugs. When the size of test case code approaches or even surpasses that of the testees, the test cases need be tested first before running them.
    • Developers will write a lot of code that will not be used by any customers. How many developers will whole-heartedly take this job? You can expect a lot more Q&D (quick-and-dirty) code than in production software.
    • QA engineers are probably not up to the task of writing or even maintaining test cases. Even if they are capable, they will not be able to modify the test cases as fast, wasting precious testing time.
  2. Java code must be compiled into class files before run. For a production software or application, this is ok because the class files will be packaged and deployed. In case of testing, the test environment may change all the time (due to limited test machines and other resources), modification to parameters is commonplace, the compilation adds an unnecessary step and hurts productivity.
  3. Java software are built with classes, interfaces and inheritance; for a well designed software, it can have elegant architecture. But for busy QA personel, how many people really have the time and capacity to handle the complexity? Who maintain the test case code?

Therefore, Java is a wrong language for general testing; designing a formal framework does not help. It will

  1. cost developers' time,
  2. add complexity to test case maintenance,
  3. slow down parameter modifications, and
  4. add more time to analyze failed test cases.

If some developer decides to use JUnit for his/her own testing, it is perfectly fine. For a team with developers and QA testers, even if all the QA testers are fully Java potent, this approach still slows down the pace and productivity and should never be adopted.

What is the solution then? Scripting! Scripting languages are designed to do many things easily, quickly and effectively; they are perfect for testing. Since they are also general-purpose programming langauges, they are up to almost anything. In fact, conceptualizing tests as just test cases and test suites may not be enough, because it limits the ways test cases be organized. For instance, I may want to connect to a test database, create a few test tables with data, then test a series of test cases before I drop all the test tables and disconnect from the database server. Such tasks are better left to the QA testers who do the testing.

Testing is a step-by-step process of execution and analysis. There is no shortcut, complicated architectures bring in little value. Powerful scripting languages are the tool for testing.

 

»»» Top «««

 

3. Java Unit Testing with JudoScript

For Java software testing, JudoScript is a perfect match for these reasons:

  1. JudoScript is a complete, general-purpose scripting language. It emphazises on doing things quickly and intuitively. At the same time, it is capable of anything that Java is capable of.
  2. JudoScript is designed ground-up for Java platforms. You can use any Java objects seamlessly in the scripts. For this reason, you can extend JudoScript by simply creating regular Java classes.
  3. JudoScript programs are run without compiling, they are easy to modify and maintain.
  4. JudoScript's rich, ready-to-use features can be used for testing itself, and are invaluable for test environment setting up and cleaning up.
  5. JudoScript's feature list is virtually unlimited because it can use any Java software packages. You can use free Java software such as jCVS, java DNS, ... or commercial Java software, or Java software developed in house.
  6. JudoScript is great not only for automated unit testing, but also for functional testing, web load testing, database application testing, XML document verification, and more.
  7. JudoScript can be used in integrated build environment such as Ant, because it supports BSF (Bean-Scripting-Framework), a standard interface for Java software to interact with scripting langauges.
  8. JudoScript has complete documentation, sample code and topical articles, on-line or hard copy. Documentation is such an important part of this language, more and more information is coming.
  9. Because JudoScript is a general-purpose programming language, it can be used beyond testing, so knowledge about JudoScript is reusable in other situations. Many propriatary testing languages only work in their own test environment. JudoScript is also an easier way to learn Java, thanks to its smooth interaction between the two.
  10. Last but not least, JudoScript is totally free. How much are you paying now?

As we mentioned earilier, there are many different set-ups for testing. The following following scheme have multiple test cases share a same pair of set-up/clean-up routines:

test environment verify and setup; // make sure the test database server
                                   // is live and make a connection.
run test case 1;
run test case 2;
run test case 3;
test environment cleanup;          // disconnect from the database
This is best modeled in JudoScript using functions:
function test_Foo_setup() {
  connect 'jdbc:oracle:thin@localhost:1521:mydb', 'orauser', 'orapass';
}
function test_Foo_cleanup() {
  disconnect();
}
function test_Foo_1() { ... }
function test_Foo_2() { ... }
function test_Foo_3() { ... }

// run the test:
test_Foo_setup();
test_Foo_1();
test_Foo_2();
test_Foo_3();
test_Foo_cleanup();

This is another scenario:

test environment verify and setup;

test set up A;
  run test case 1;
test clean up A;

test set up A;
  run test case 2;
test clean up A;

test set up B;
  run test case 3;
test clean up A;

run test case 4;

test environment cleanup;

In addition to the global setup/cleanup, many test cases have their own environment; some of the test cases share a same pair of setup/cleanup routines. In this case, the inheritance feature of object-oriented programming helps to orginize code nice and clean:

function test_Foo_setup() { ... }
function test_Foo_cleanup() { ... }

class Test_Base
{
  function setup()   {} // empty
  function cleanup() {} // empty
  function doTest()  {} // empty -- should be overridden

  function runTest() {
    setup();
    doTest();
    cleanup();
  }
}

class Test_Foo_Group_A extends Test_Base
{
  function setup()   { /* group A setup */ }
  function cleanup() { /* group A cleanup */ }
  function doTest()  {} // empty -- should be overridden
}

class Test_Foo_Group_B extends Test_Base
{
  function setup()   { /* group B setup */ }
  function cleanup() { /* group B cleanup */ }
  function doTest()  {} // empty -- should be overridden

class Test_Foo_1 extends Test_Foo_Group_A
{
  construtor { desc = "Project Foo's test case 1."; }
  function doTest { /* test case 1 code */ }
}

class Test_Foo_2 extends Test_Foo_Group_A
{
  construtor { desc = "Project Foo's test case 2."; }
  function doTest { /* test case 2 code */ }
}

class Test_Foo_3 extends Test_Foo_Group_B
{
  construtor { desc = "Project Foo's test case 3."; }
  function doTest { /* test case 3 code */ }
}

class Test_Foo_4 extends Test_Base
{
  construtor { desc = "Project Foo's test case 4."; }
  function doTest { /* test case 4 code */ }
}

// Run the tests
test_Foo_setup();
(new Test_Foo_1).runTest();
(new Test_Foo_2).runTest();
(new Test_Foo_3).runTest();
(new Test_Foo_4).runTest();
test_Foo_cleanup();

The class hierarchy is like this:

Test_Base
    Test_Foo_Group_A
        Test_Foo_1
        Test_Foo_2
    Test_Foo_Group_B
        Test_Foo_3
    Test_Foo_4

In effect, the test cases are grouped; each group shares a pair of setup() and cleanup() methods. You get a nicely laid out paradigm. The cost is house-keeping code and complexity. A by-product is, you can bundle more information (such as descriptions) with the test cases.

The second scenario can be implement with functions as well. The point is, JudoScript is flexible enough for you to lay out tests in the most appropriate way for the situation. JudoScript language has built-in assertion and exception handling facility. It also has standard, error and log output streams. If you have some special reporting needs, you can easily write your own report functions, perhaps using the the flexible print and println statements in JudoScript. JudoScript scripts can include other scripts; and external programs can be run from another script. There are many ways you can organize your test suites.

To truly understand JudoScript language and feel its power, there is no other way than reading featured articles such as Introduction to JudoScript, and try it out by yourself. JudoScript is an easy-to-use and easy-to-learn language; even if you have never programmed in Java, you won't have any problem start writing useful JudoScript code.

 

»»» Top «««

 

4. Code Listings

  1. SCP Test Case 1
  2. SCP Test Case 2




Copyright © 2001-2005 JudoScript.COM. All Rights Reserved.