Industry musings

Since I started a new blog for the work I do from time to time on DUIM, I've decided to repurpose this blog to keep notes on things I encounter professionally. For most Lispers that is going to make this blog very uninteresting indeed, since the technology I use mainly at work is Java based.

I've been developing in Java since version 1.0 if I remember correctly (self-taught from the brown nutshell book with the guys doing semaphore on the front). Since then I've developed distributed Corba components, AWT (pre Swing!) and Swing applications, Web and eCommerce solutions, several enterprise applications and tools, through to my current role developing a Knowledge Management solution, all in Java.

... so I can update this older blog with the current address of my (ahem) newer blog focussing on DUIM. Find it at http://cl-duim.blogspot.com.

... but not much. This is more a hint to remind me of things to be careful of than anything else.

The test results are up to:


Ran 7125 checks in 110 tests from 14 suites.
42.857143% (6 / 14) suites executed correctly.
70.90909% (78 / 110) tests executed correctly.
92.98246% (6625 / 7125) checks executed correctly.


I have no idea where the additional tests (up from 7080) have come from...

Most of the additional passes have come from changing 'CHECK-EQUAL' tests (which in the Dylan made use (I presume) of Dylan's generic '=' function, but which in the Lisp version invokes 'EQUAL' instead) to 'CHECK' and passing #'dylan-= as the functional argument.

At least now colours and space-requirement objects etc. compare as equivalent when they should (which they often didn't under EQUAL).

Next I'm going to look at all the scrolling and layout failures and hopefully knock some of those (most?) on the head.

I found a problem in the test harness code (expected-column-named-width wasn't returning a value!) which has improved the test results a little. The results are now:


Ran 7080 checks in 110 tests from 14 suites.
21.428572% (3 / 14) suites executed correctly.
64.545456% (71 / 110) tests executed correctly.
91.77966% (6498 / 7080) checks executed correctly.


The main culprits accounting for the failing tests are (approx values):

• LTRB tests -- 60 fails


• Layout tests -- 130 fails


• Gadget tests -- 100 fails


• Scrolling tests -- 275 fails

The other 15 or so fails are spread over various tests. Obviously I'm going to focus my attention on the 'big 4' mentioned above. I think when I get to 95% passing I will probably move on to focus on porting a back end properly. That leaves around 230 defects that need resolving before I can move on.

... and, for the most part, running. The count at the moment is as follows:


Ran 7080 checks in 110 tests from 14 suites.
21.428572% (3 / 14) suites executed correctly.
64.545456% (71 / 110) tests executed correctly.
88.418076% (6260 / 7080) checks executed correctly.


A couple of the tests originally present have been commented out (but only 4-5), so the figure represented above isn't quite fully-representative but I think 88% isn't bad for a first run.

Most of the failures are related to the scrolling and layout tests; hopefully there's some low-hanging fruit that will improve the figures easily to be found in those areas.

It's also worth noting that the current tests miss out some of the DUIM functionality; but I'm hopeful that once any easy fixes are put in, the port should be ready for a back end at last.

Well, it's been a while. I'm still working on DUIM, on and off. I'm going through an 'on' period at the moment, for better or worse :-)

I've 'fixed' some of the issues I was having with the code; in particular I've implemented a work-around for the Dylan methods that specialise on subclass(<class>) using metaclasses. Basically each class defined via define-dylan-class also gets its own metaclass defined (so the metaclass hierarchy mirrors the 'concrete' class hierarchy).

I don't like this approach :-)

In the gadget-text-parser and gadget-value-printer methods I've made use of class-prototype objects instead which is a slight improvement.

I think the best solution will be to work out how to create new CLOS specializer classes; implementations provide EQL specializers and I guess it should be possible to implement a SUBCLASS specializer. I'll have to look into this.

The code is still confused regarding the use of lists and vectors, but on the whole the codebase at least hangs together in this regard. This needs revisiting at some point in the future. It might be nice to implement (or find, it's possible something exists already) a generic CLOS-based collections package (I can see the value of operating on lists / arrays via different methods, for efficiency reasons -- but I really like the Dylan approach that wraps all collection types up in a single API).

I've stopped working on any back-end code at the moment. My priority is in getting the fun-O tests to run (not necessarily pass!). At the moment around 2k of them run, and about 85% pass so I'm nearly there with this, I think. It does seem that some of the tests target a different version of DUIM than the code I have though; the most obvious case of this is in the tests for text-styles, which don't pass a name to the methods. All the text-style code in DUIM appears to require a name (as well as a family). The documentation I've seen supports the tests, but the documentation might be out of date. There are other examples too (<TEST-STATUS-BAR-PANE> inherits from <LEAF-PANE> but the status-bar handling methods want to (setf (sheet-children bar) ...); this works in DUIM since DUIM's status bar inherits from <ROW-LAYOUT> which includes <MULTIPLE-CHILD-MIXIN> -- but if the DUIM code was always organised the way it is at the moment, the status bar tests can never have worked.

I guess I'll get there in the end.

My plan of attack at the moment is:

• Get the fun-o tests running, and fix any low-hanging fruit to get as many passing as possible. At this point, DUIM-CORE, DUIM-TESTS and DUIM-EXTENDED will be completely ported, barring errors (I don't think there are tests for the extensions for example)


• Write a back-end. I'll probably port the existing win32 back end since at the moment that's the platform on which I'm doing most development on this. I want to see something happening on-screen now...


• Port the examples and demos -- and write a couple more


• Port DEUCE

I think there may be some light appearing at the end of the tunnel :-)

Ok, once again I've been slacking rather when it comes to updating this blog. Apologies for watchers, but I have a bit of an excuse this time -- namely that my family has been expanded by a new arrival, Isaac Arthur Rose, who was born at 7.15 am on the 10th of June. Needless to say I've been a little distracted with that.

There has been some progress with DUIM. I initially decided to write unit tests using the RT framework (which I did, up to a point). Over the last couple of weeks I've converted the FunDev tests to Lisp, and written a very simple equivalent of FunDev's 'TestWorks' harness to reduce the changes to the FunDev tests that needed to be made. Since there were a couple of problems using RT for me (difficult to test intermediate states, since a bunch of the DUIM code isn't functional) I figured I'd use this quick hack for all my tests, and have now also moved my unit tests over to this new framework.
Currently, there's a bunch of code that fails the tests (and the converted Dylan tests don't yet compile) but I feel I'm making (good) progress at the moment.

Stuff to do:
o My unit-test framework is very bad at running multiple suites, and could do a better job of reporting its results. I need to give it a little polish, document it, and then perhaps release it on its own (oh joy, yet another test framework!). It also needs to do better at handling errors in the test setup / teardown code (this work is currently done outside the framework altogether, so failing code kills the test run which is not ideal.
o I need to formalise (and document) a strategy for object creation within DUIM. Currently I'm specialising MAKE-INSTANCE by EQL specialising on STANDARD-CLASS which may or may not be legal (there was a thread on c.l.l. about this recently, dig out the reference). Note also that DUIM occasionally has a MAKE method for subclasses of some type, rather than for specific types. The Dylan tests mentioned above also do this, so I need to come up with something reasonable in the near future.
o The code is currently quite schizophrenic over its handling of vectors and lists. Dylan is more forgiving than Lisp when it comes to dealing with these types (for example, it appears to be legal to give APPLY a vector as its final argument) and parts of the DUIM code rely on this. The transliteration currently uses the same types as the original Dylan code so I've hit a few places where vectors need to be converted to lists in order for the code to work (the current example is how regions are held in and friends). This should just be a matter of sitting down and going through the code (again, trying to be consistent).
o I've tried to avoid 'Lispifying' the code to date (it still looks very similar to the Dylan code). I think I've gone about as far as I can without doing at least a little 'Lispification' (particularly wrt slot accessors), so some of that needs doing too.
o I need to document more, especially if the code starts to diverge from the Dylan implementation since the FunDev documentation will not be so useful anymore.

I think this needs prioritising in the following order: list/vector changes, unit test changes, object construction, lispification with documentation being a common thread over all tasks.

More updates soon (I'm not going to promise since I'm so bad at providing them, but I will promise to try harder :-)

# posted by Duncan Rose @ 2:42 pm 0 comments

Well, 'real life' kind of caught up with me for a while after the new year. I haven't died (yet) or been run over by a bus or anything and I'm back looking at the DUIM port (I was intending to take a break for a week, and here we are four months later...).

I think the break may have been useful, I see some holes in the code that weren't apparent to me before (but then that might be an excuse to concentrate on the interesting side of understanding the windowing model better etc., rather than to actually, you know, get it working).

I'll try to make a set time for posting here in future (maybe Friday afternoons each week) to provide regular progress reports if for no other reason than to highlight any lack of progress to myself so I'll be spurred on to work harder :-)

Things are starting to come together wrt DUIM. I've started to work on a back end (using LTK at the underlying toolkit) and converted over the tic-tac-toe example program from the Dylan.

No luck so far getting things to run; I've made too many arbitrary and inconsistent conversions throughout the code (mainly regarding when to use lists and when to use vectors); the Dylan DUIM code has a bunch of slots or variables bound to (for example) #(). To a Lisp programmer this looks like a vector, so these fields have been made vectors in the Lisp. Often the usage of the object demands a list however, so I'm getting a few incompatible type exceptions being thrown. The opposite situation is also a problem.

It's just a matter of time until these are all sorted out though (and hopefully not a huge amount of time either).

Also there is a distinct lack of documentation regarding what needs to be implemented for the back end. Until I sit down and go through it properly, and document (which would probably be quicker, but not as much fun as hacking things until they work) there's a good deal of trial and error. Knowing something about McCLIM back ends simplifies things somewhat, but not as much as I'd like!

The immediate TODO is currently:

1. Get the core, back end and tic-tac-toe code to a point where the tic-tac-toe demo works
2. Convert the code to a Lispier style (remove DEFINE-DYLAN-CLASS et al)
3. More unit tests
4. Convert the rest of the example programs, fix anything necessary to get them running
5. Convert the Dylan DUIM tests
6. Examples, documentation, applications
7. More back ends (I'm keen to bash out a "better" Beagle -- perhaps this will lead to improvements in the McCLIM Beagle too)

Somewhere during that list I'll make a release -- hopefully this will be "soon" but I don't feel the code is there just yet. Releasing now would be confusing I think.

I am pretty happy with the state of things as they stand, but will be happier when everything's moved on a way.

So forward with the debugging!