(new) RE: lattice drawing program --Bob McElrath?, Thu, 22 Jul 2004 09:56:24 -0700 reply
Awesome! I will be travelling to conferences for the next month, so I'm not sure I will have time to work on this again until the end of August. :(

You should let the zwiki folks know of your repository, so they can pull your patches into the next release. They are very regular with monthly releases.

Bill Page [bill.page1@sympatico.ca]? wrote:

Bob,

I think I now understand how the darcs repository works, so I have set up two repositories for the AxiomWiki? project that you can get to from here:

One contains the patches to LatexWiki to include Axiom, the other contains minor patches to Zwiki.

Please let me know if you can access these or if I have done something wrong in using darcs.

I am starting to like darcs, so if everything is ok, with Tim Daly's approval I might setup a repository here also for Axiom itself. Ideally, I would like to link this Axiom repository to changes that might be made to Axiom library code online from the Axiom Wiki.

Thanks.

Bill Page.

Re: literate programming pamphlet files forMathAction --Bill Page, Fri, 01 Oct 2004 06:45:27 -0700 reply
On Wednesday, September 29, 2004 3:10 PM Bob McElrath? wrote:

... Bill Page wrote:
... There are few more notes and some examples here:

My approach is to use Norman Ramsey's Latex to HTML (l2h) noweave filter. This filter together with noweave, is able to produce HTML files directly from noweb (pamphlet) files.

...

Let me suggest that you do this as a ZWiki "pagetype". (see pagetypes/ in the ZWiki source) The relevant code for latexwiki is in the __init__.py file.

Yes, this is on my to-do-right list :)

... Let me tell you what I have been doing, since it is very similar...

... I have started [rewrite of ZWiki page-rendering code]? this, and right now have a standalone code which does it "properly". This is a subclass of StructuredText and uses that machinery to render pages.

Why this is relevant to you:

When this is done I will start adding new page types. For me the first will be "true" latex.

Wow, from what I have seen in Norman Ramsey's l2h code, that sounds like a lot of work to do "from the ground-up", so as to speak. BTW, if you read the l2h code, you will notice that it does not use any regular expression pattern matching stuff at all, but a rather different and seemingly robust, mostly declarative parsing method that I hadn't seen before called "continuations" - a little like co routines but different. The syntax and semantics of Icon is very high level - not so different from Python, so in principle a conversion of l2h to Python might not be out of the question. In any case, before embarking on "true" latex, I think it would be worth looking at Ramsey's l2h documentation. See

But since this is all done within the StructuredText system, new pagetypes (such as "spad") can inherit the needed latex functionality from latexwiki. Thus we can combine our work into new page types in ways that cannot mangle the document.

I am not so sure how possible it might be to use Ramsey's approach in the StructuredText framework - not such a good match I think. One of my goals here is code re-use and to avoid writing much new code except that needed as "glue".

The StructuredText classes are very simple. Basically for each text "object" (like $math$) you have to write one function to recognize it and one function to render it. In addition there is a list of which "recognizers" have more importance than others, so you know what to do with things like [foo $bar$]? vs. $foo [bar]?$ or -- which takes precedence, the link [] or the latex? This is very difficult to do properly with successive regexes that operate on the whole document. That it works at all right now is a rather large hack involving successive conversions to html, and ignoring html...

For me, such re-used has only really begun to seem possible and practical in the new world of supercomputer desktops. I used to think badly of, and label as a "hack", the sometimes inefficient and often conceptually muddled software that results from such "cross-breeding", but the technology has definitely changed since the last time I did this kind of work...

This is not as good as a formally defined document grammar, but is probably the best we can get with what we are doing.

Join me on #zwiki at irc.freenode.net if you'd like to discuss this further. I think we should discuss the "big scheme" before we head in different directions and end up with a pile of incompatible software.

I would like to discuss this further, but I find synchronous communication via irc etc. to awkward given my irregular schedule. Let's continue here or via wiki somewhere (your place or mine? :)

I have placed my preliminary StructuredText classes here:

   http://bob.mcelrath.org/WikiStructuredText


along with a standalone example program showing how to use it. I hope this should make it obvious how to extend.

This will result in a more extensible, understandable, and robust codebase. Not to mention faster.

Thanks. I will look at this but I can not promise that it will be soon.

P.S. do I understand that MathAction is your website, and not a piece of software?

Yes. Specifically MathAction is what I call the stand-alone LatexWiki-based website at

As you know, there is also the Axiom Portal web site that is Plone-based and supports LatexWiki objects.

I really don't want to get into maintaining a piece of software. I would much prefer to leave that to other people (such as yourself). But as seems common among "open source types", I am a bit of a maverick when it comes to using (and re-using) software, so I can't guarantee that I will always be working in a structured mode that will be easy to merge. As usual, our ambitions still far out strip our resources and time constraints...

Regards, Bill Page.

pamphlet support on MathAction --Bob McElrath?, Tue, 12 Oct 2004 10:22:30 -0700 reply
Bill Page [bill.page1@sympatico.ca]? wrote:

Martin,

I am sorry that I do not have anything integrated with MathAction yet. I ran into a few problems with my modifications of the l2h (latex to html) noweave -filter and am still making changes to fix this before adding it to LatexWiki.

In some cases the internal HTML linkages created by noweave mess up the selected pass thru of the Latex coding like and \begin{axiom} because noweave assume that the output of the l2h filter will be pure html with nothing "strange" embedded like LaTeX? coding. The LatexWiki html+latex document type is a little strange in that it permits (expects?) the LaTeX? coding just to be inserted in the HTML even though the the syntax of these two languages is not really compatible.

I have been rewriting the StructuredText document type in the last week or so, and have decided that html+latex must go away because these two languages are not compatible. Specifically, latex can contain less-than and greater-than symbols, and it is not possible to disentangle latex from html tags.

## I have decided on the following "new" pagetypes:

• stx + latex -- cannot contain html, latex is "higher-priority" so that e.g. [$a$]? is a wiki link with an equation and $[hackedejder.htm]?$ is an equation with brackets in it (and not a wikilink). There will also be a backend which renders mini-latex (below) to generate printable (ps/pdf) output.
• mini-latex -- just enough latex markup to get stx functionality, and will render if passed to latex, but will actually be processed by the StructuredText machinery as a new pagetype. i.e. it will know \begin{itemize}, \item, \cite, \tt, \bf, \it, etc and not require a preamble.
• latex -- true latex, will be passed to latex2html or tth for the web, pdflatex for printable output.

It is only possible to mix html and latex if we require that latex escape the characters <>&. This seems very difficult, especially for hand-coded latex, but maybe your generator can do it? I am debating whether stx+latex needs a true escaping mechanism. I think we can only retain html+latex if we require the embedded latex to be escaped properly.

The hyperref and hypextra packages allow one to embed HTML in latex but require latex-style tags like \htmladdimg. (see also tex4ht)

My latest work is here:

    http://bob.mcelrath.org/WikiStructuredText


right now it duplicates the ZWiki functionality. (These classes will be added to ZWiki and replace stx.py) I have not added latex yet. I hope by looking at WikiDocument?.py and WikiHTML?.py it is obvious what to do.

I have much work to do here before getting to latexwiki
like integrating the WikiLink mechanism and merging with ZWiki.
There are also related problems about how l2h likes to convert \ref{} to equation \label{} versus the way that LatexWiki is doing it now. LatexWiki insists on renumbering equations and that is not compatible with the links that were earlier inserted by l2h. Plus the insertion of html anchors for the equations is messed up by the way that I coded the pass thru of LaTeX? coding.

I want to use \ref and \label instead. Hopefully this will make it in the next release. But if you beat me to making a patch for this, please send it along!

I think what I need to do is have the l2h filter wrap the LaTeX? in a special "html-like" tag e.g.

$$...$$

Some paragraph

\begin{axiom} ... \end{axiom}

Then (hopefully) the rest of the noweave indexing mechanism wont insist on inserting html inside these html tags.

Even this is treacherous. [hackedejder.htm]? mark of a good markup system is that it has the proper escaping to write a document about itself. It is obtuse, but can you place the string "" inside a latex block? It looks like hypextra does what you propose, but it is incapable of placing inside a latex block.

If we can think of nothing better, I would accept enclosing latex-in-html with tags. However I think a better idea would be and tags.

Anyway, I will be away at a meeting (and short vacation) for the next 10 days so don't expect much progress until later this month.

Okay. ;) Well, let's see how much I've gotten done when you get back.

P.S. this message will hose ZWiki and LatexWiki because it contains mock-tags. I propose also that mail-in messages be stx without html. This is required anyway if we mark-up message quoting with >. i.e. if the mime-type of the message is text/plain we treat it as text/stx (since no mailer will generate text/stx). Alternatively if the email contains mime multipart/alternative with an HTML section, we can use that directly. This points toward comments being independent documents of the parent -- and each comment could have a different pagetype!

pamphlet support on MathAction --Bill Page, Wed, 13 Oct 2004 10:29:40 -0700 reply
Martin,

On Wednesday, October 13, 2004 6:28 AM you wrote:

Bill Page [bill.page1@sympatico.ca]? wrote:
Bill Page [bill.page1@sympatico.ca]? wrote: > Martin, > > I am sorry that I do not have anything integrated with > MathAction yet. I ran into a few problems with my > modifications of the l2h (latex to html)noweave -filter > and am still making changes to fix this before adding it to LatexWiki.

I do understand now that it is indeed a major effort. In fact, I was quite astonished about your approach: I would have added a "pamphlet" style that accepts pamphlet files only, generates the latex via the document command, then the html via l2h or latex2html or hevea or whatever.

But that is almost what I am doing. What you call document is just noweave with a couple of Axiom specific options. l2h is called as a filter from inside noweave to do the html conversion. But l2h does not do a reasonable job of converting equations to html. So what I decided was to use those features of LatexWiki that already do essentially just this part of the LaTeX? conversion alone. The output of my modified l2h filter is just the HTML+Latex format of LatexWiki.

(hevea would be an especially nice option: the pictures take a long time to load)

It is true that the LaTeX? generated images take a comparatively long time to load. But most people seem to think that hevea does a pretty awful job of converting LaTeX? equations to HTML. The fact that it (sort of) does it at all is a kind of miracle since HTML was not designed to display mathematics. That is the function of the mathML extension of the language. The best why would be to convert LaTeX? equations to mathML. But mathML is still not supported widely and might not really be identical to the LaTeX? generated result that everyone expects.

So, in short, I think that Bob's approach is the one to go.

I agree in the long run. But it will take some time to get there and then to add the Axiom and Reduce functionality etc. Another few days of my time (when I get time) will probably be all that is needed to get the current approach working.

By the way: also the dollar-sign poses problems in StructuredText+LaTeX? and has to be escaped in single-quoted text...

Yes. But not such special escaping will be required in the pamphlet files themselves - it is only necessary internally between the l2h filter and the LatexWiki HTML+Latex format.

Regards, Bill Page.