Monday, February 05, 2007
More testing progress ...
... but not much. This is more a hint to remind me of things to be careful of than anything else.
The test results are up to:
Ran 7125 checks in 110 tests from 14 suites.
42.857143% (6 / 14) suites executed correctly.
70.90909% (78 / 110) tests executed correctly.
92.98246% (6625 / 7125) checks executed correctly.
I have no idea where the additional tests (up from 7080) have come from...
Most of the additional passes have come from changing 'CHECK-EQUAL' tests (which in the Dylan made use (I presume) of Dylan's generic '=' function, but which in the Lisp version invokes 'EQUAL' instead) to 'CHECK' and passing #'dylan-= as the functional argument.
At least now colours and space-requirement objects etc. compare as equivalent when they should (which they often didn't under EQUAL).
Next I'm going to look at all the scrolling and layout failures and hopefully knock some of those (most?) on the head.
The test results are up to:
Ran 7125 checks in 110 tests from 14 suites.
42.857143% (6 / 14) suites executed correctly.
70.90909% (78 / 110) tests executed correctly.
92.98246% (6625 / 7125) checks executed correctly.
I have no idea where the additional tests (up from 7080) have come from...
Most of the additional passes have come from changing 'CHECK-EQUAL' tests (which in the Dylan made use (I presume) of Dylan's generic '=' function, but which in the Lisp version invokes 'EQUAL' instead) to 'CHECK' and passing #'dylan-= as the functional argument.
At least now colours and space-requirement objects etc. compare as equivalent when they should (which they often didn't under EQUAL).
Next I'm going to look at all the scrolling and layout failures and hopefully knock some of those (most?) on the head.
91%+ of tests passing...
I found a problem in the test harness code (
Ran 7080 checks in 110 tests from 14 suites.
21.428572% (3 / 14) suites executed correctly.
64.545456% (71 / 110) tests executed correctly.
91.77966% (6498 / 7080) checks executed correctly.
The main culprits accounting for the failing tests are (approx values):
The other 15 or so fails are spread over various tests. Obviously I'm going to focus my attention on the 'big 4' mentioned above. I think when I get to 95% passing I will probably move on to focus on porting a back end properly. That leaves around 230 defects that need resolving before I can move on.
expected-column-named-width wasn't returning a value!) which has improved the test results a little. The results are now:Ran 7080 checks in 110 tests from 14 suites.
21.428572% (3 / 14) suites executed correctly.
64.545456% (71 / 110) tests executed correctly.
91.77966% (6498 / 7080) checks executed correctly.
The main culprits accounting for the failing tests are (approx values):
- LTRB tests -- 60 fails
- Layout tests -- 130 fails
- Gadget tests -- 100 fails
- Scrolling tests -- 275 fails
The other 15 or so fails are spread over various tests. Obviously I'm going to focus my attention on the 'big 4' mentioned above. I think when I get to 95% passing I will probably move on to focus on porting a back end properly. That leaves around 230 defects that need resolving before I can move on.
Saturday, February 03, 2007
Tests Building...
... and, for the most part, running. The count at the moment is as follows:
A couple of the tests originally present have been commented out (but only 4-5), so the figure represented above isn't quite fully-representative but I think 88% isn't bad for a first run.
Most of the failures are related to the scrolling and layout tests; hopefully there's some low-hanging fruit that will improve the figures easily to be found in those areas.
It's also worth noting that the current tests miss out some of the DUIM functionality; but I'm hopeful that once any easy fixes are put in, the port should be ready for a back end at last.
Ran 7080 checks in 110 tests from 14 suites.
21.428572% (3 / 14) suites executed correctly.
64.545456% (71 / 110) tests executed correctly.
88.418076% (6260 / 7080) checks executed correctly.
A couple of the tests originally present have been commented out (but only 4-5), so the figure represented above isn't quite fully-representative but I think 88% isn't bad for a first run.
Most of the failures are related to the scrolling and layout tests; hopefully there's some low-hanging fruit that will improve the figures easily to be found in those areas.
It's also worth noting that the current tests miss out some of the DUIM functionality; but I'm hopeful that once any easy fixes are put in, the port should be ready for a back end at last.
