html2wml (1)

Html2Wml can be used as either a shell command:

  $ html2wml file.html

or as a \s-1CGI:\s0

/cgi-bin/html2wml.cgi?url=/index.html In both cases, the file can be either a local file or a \s-1URL\s0.

Html2Wml converts \s-1HTML\s0 pages to \s-1WML\s0 decks, suitable for being viewed on a Wap device. The program can be launched from a shell to statically convert a set of pages, or as a \s-1CGI\s0 to convert a particular (potentially dynamic) \s-1HTML\s0 resource.

Althought the result is not guarantied to be valid \s-1WML\s0, it should be the case for most pages. Good \s-1HTML\s0 pages will most probably produce valid \s-1WML\s0 decks. To check and correct your pages, you can use W3C's softwares: the \s-1HTML\s0 Validator, available online at http://validator.w3.org and \s-1HTML\s0 Tidy, written by Dave Raggett.

Html2Wml provides the following features:

\$1

translation of the links limitation of the cards size by splitting the result into several cards inclusion of files (similar to the \s-1SSI\s0) compilation of the result (using the \s-1WML\s0 Tools, see the section on "LINKS") a debug mode to check the result using validation functions

Please note that most of these options are also available when calling Html2Wml as a \s-1CGI\s0. In this case, boolean options are given the value \*(L"1\*(R" or \*(L"0\*(R", and other options simply receive the value they expect. For example, \*(C`--ascii\*(C' becomes \*(C`?ascii=1\*(C' or \*(C`?a=1\*(C'. See the file t/form.html for an example on how to call Html2Wml as a \s-1CGI\s0.

\$1

When this option is on, named \s-1HTML\s0 entities and non-ASCII characters are converted to \s-1US-ASCII\s0 characters using the same 7 bit approximations as Lynx. For example, \*(C`©\*(C' is translated to \*(L"(c)\*(R", and \*(C`ß\*(C' is translated to \*(L"ss\*(R". This option is off by default. This option tells Html2Wml to collapse redundant whitespaces, tabulations, carriage returns, lines feeds and empty paragraphs. The aim is to reduce the size of the \s-1WML\s0 document as much as possible. Collapsing empty paragraphs is necessary for two reasons. First, this avoids empty screens (and on a device with only 4 lines of display, an empty screen can be quite ennoying). Second, Html2wml creates many empty paragraphs when converting, because of the way the syntax reconstructor is programmed. Deleting these empty paragraphs is necessary like cleaning the kitchen :-) If this really bother you, you can desactivate this behaviour with the \*(--nocollapse option. This option tells Html2Wml to completly ignore all image links. This option tells Html2Wml to replace the image tags with their corresponding alternative text (as with a text mode web browser). This option is on by default. This option is on by default. This makes Html2Wml flattens the \s-1HTML\s0 tables (they are linearized), as Lynx does. I think this is better than trying to use the native \s-1WML\s0 tables. First, they have extremely limited features and possibilities compared to \s-1HTML\s0 tables. In particular, they can't be nested. In fact this is normal because Wap devices are not supposed to have a big \s-1CPU\s0 running at some zillions-hertz, and the calculations needed to render the tables are the most complicated and CPU-hogger part of \s-1HTML\s0. Second, as they can't be nested, and as typical \s-1HTML\s0 pages heavily use imbricated tables to create their layout, it's impossible to decide which one could be kept. So the best thing is to keep none of them. [Note] Although you can desactivate this behaviour, and although there is internal support for tables, the unlinearized mode has not been heavily tested with nested tables, and it may produce unexpected results. This option tells Html2wml to convert all non-ASCII characters to numeric entities, i.e., "e\*'" becomes \*(C`é\*(C', and "\*8" becomes \*(C`ß\*(C'. By default, this option is off. This options tells Html2Wml not to use the <pre> tag. This option was added because the compiler from \s-1WML\s0 Tools 0.0.4 doesn't support this tag. This options sets the template that will be used to reconstruct the \*(C`href\*(C'-type links. See the section on "LINKS RECONSTRUCTION" for more information. This option sets the template that will be used to reconstruct the \*(C`src\*(C'-type links. See the section on "LINKS RECONSTRUCTION" for more information. This option allows you to limit the size (in bytes) of the generated cards. Default is 1,500 bytes, which should be small enought to be loaded on most Wap devices. See the section on "DECK SLICING" for more information. This option sets the threshold of the split event, which can occur when the size of the current card is between \*(C`max-card-size\*(C' - \*(C`card-split-threshold\*(C' and \*(C`max-card-size\*(C'. Default value is 50. See the section on "DECK SLICING" for more information. This options sets the label of the link that points to the next card. Default is \*(L"[>>]\*(R", which whill be rendered as "[>>]". This options sets the label of the link that points to the previous card. Default is \*(L"[<<]\*(R", which whill be rendered as "[<<]". Use this option to set the username for an authenticated request. Use this option to set the password for an authenticated request. Use this option to activate proxy support. By default, proxy support is activated. See the section on "PROXY SUPPORT". Setting this option tells Html2Wml to use the compiler from \s-1WML\s0 Tools to compile the \s-1WML\s0 deck. If you want to create a real Wap site, you should seriously use this option in order to reduce the size of the \s-1WML\s0 decks. Remember that Wap devices have very little amount of memory. If this is not enought, use the splitting options. Take a look in wml_compilation/ for more information on how to use a \s-1WML\s0 compiler with Html2Wml. Use this option (in shell mode) to specify an output file. By default, Html2Wml prints the result to standard output. This option activates the debug mode. This prints the output result with line numbering and with the result of the \s-1XML\s0 check. If the \s-1WML\s0 compiler was called, the result is also printed in hexadecimal an ascii forms. When called as a \s-1CGI\s0, all of this is printed as \s-1HTML\s0, so that can use any web browser for that purpose. When this option is on, it send the \s-1WML\s0 output to \s-1XML:\s0:Parser to check its well-formedness.

The deck slicing is a feature that Html2Wml provides in order to match the low memory capabilities of most Wap devices. Many can't handle cards larger than 2,000 bytes, therefore the cards must be sufficiently small to be viewed by all Wap devices. To achieve this, you should compile your \s-1WML\s0 deck, which reduce the size of the deck by 50%, but even then your cards may be too big. This is where Html2Wml comes with the deck slicing feature. This allows you to limit the size of the cards, currently only before the compilation stage. On some Wap phones, slicing the deck is not sufficient: the \s-1WML\s0 browser still tries to download the whole deck instead of just picking one card at a time. A solution is to slice the \s-1WML\s0 document by decks. See the figure below.

_____________ _____________ | deck | | deck #1 | | _________ | | _________ | | | card #1 | | | | card | | | |_________| | | |_________| | | _________ | |_____________| | | card #2 | | | |_________| | . . . | _________ | | | ... | | _____________ | |_________| | | deck #n | | _________ | | _________ | | | card #n | | | | card | | | |_________| | | |_________| | |_____________| |_____________| WML document WML document sliced by cards sliced by decks What this means is that Html2Wml generates several \s-1WML\s0 documents. In \s-1CGI\s0 mode, only the appropriate deck is sent, selected by the id given in parameter. If no id was given, the first deck is sent. Currently, Html2Wml estimates the size of the card on the fly, by summing the length of the strings that compose the \s-1WML\s0 output, texts and tags. I say \*(L"estimates\*(R" and not \*(L"calculates\*(R" because computing the exact size would require many more calculations than the way it is done now. One may objects that there are only additions, which is correct, but knowing the exact size is not necessary. Indeed, if you compile the \s-1WML\s0, most of the strings of the tags will be removed, but not all.

For example, take an image tag: \*(C`<img src="images/dog.jpg" alt="Photo of a dog">\*(C'. When compiled, the string \*(C`"img"\*(C' will be replaced by a one byte value. Same thing for the strings \*(C`"src"\*(C' and \*(C`"alt"\*(C', and the spaces, double quotes and equal signs will be stripped. Only the text between double quote will be preserved... but not in every cases. Indeed, in order to go a step further, the compiler can also encode parts of the arguments as binary. For example, the string \*(C`"http://www."\*(C' can be encoded as a single byte (\*(C`8F\*(C' in this case). Or, if the attribute is \*(C`href\*(C', the string \*(C`href="http://\*(C' can become the byte \*(C`4B\*(C'.

As you see, it doesn't matter to know exactly the size of the textual form of the \s-1WML\s0, as it will always be far superior to the size of the compiled form. That's why I don't count all the characters that may be actually written.

Also, it's because I'm quite lazy ;-) If you intent to create real \s-1WML\s0 pages, you should really consider to always compile them. If you're not convinced, here is an illustration.

Take the following \s-1WML\s0 code snipet:

<a href='http://www.yahoo.com/'>Yahoo!</a> It's the basic and classical way to code an hyperlink. It takes 42 bytes to code this, because it is presented in a human-readable form.

The \s-1WAP\s0 Forum has defined a compact binary representation of \s-1WML\s0 in its specification, which is called \*(L"compiled \s-1WML\s0\*(R". It's a binary format, therefore you, a mere human, can't read that, but your computer can. And it's much faster for it to read a binary format than to read a textual format.

The previous example would be, once compiled (and printed here as hexadecimal):

1C 4A 8F 03 y a h o o 00 85 01 03 Y a h o o ! 00 01 This only takes 21 bytes. Half the size of the human-readable form. For a Wap device, this means both less to download, and easier things to read. Therefore the processing of the document can be achieved in a short time compared to the tectual version of the same document.

There is a last argument, and not the less important: many Wap devices only read binary \s-1WML\s0.

Actions are a feature similar to (but with far less functionalities!) the \s-1SSI\s0 (Server Side Includes) available on good servers like Apache. In order not to interfere with the real \s-1SSI\s0, but to keep the syntax easy to learn, it differs in very few points. Basically, the syntax to execute an action is:

<!-- [action param1="value" param2='value'] --> Note that the angle brackets are part of the syntax. Except for that point, Actions syntax is very similar to \s-1SSI\s0 syntax. Only few actions are currently available, but more can be implemented on request.

Includes a file in the document at the current point. Please note that Html2Wml doesn't check nor parse the file, and if the file cannot be found, will silently die (this is the same behavior as \s-1SSI\s0). \*(C`virtual=url\*(C' \*(-- The file is get by http. \*(C`file=path\*(C' \*(-- The file is read from the local disk.

Returns the size of a file at the current point of the document. \*(C`virtual=url\*(C' \*(-- The file is get by http. \*(C`file=path\*(C' \*(-- The file is read from the local disk. If you use the file parameter, an absolute path is recommend.

Skips everything until the first \*(C`end_skip\*(C' action.

The following parameters can be used for any action. This paramater restricts the action for the given output format. Currently, the only available format is "\*(C`wml\*(C'" (when using \*(C`html2chtml\*(C' the format is "\*(C`chtml\*(C'"). If you want to share a navigation bar between several \s-1WML\s0 pages, you can \*(C`include\*(C' it this way:

<!-- [include virtual="nav.wml"] --> Of course, you have to write this navigation bar first :-)

If you want to use your current \s-1HTML\s0 pages for creating your \s-1WML\s0 pages, but that they contains complex tables, or unecessary navigation tables, etc, you can simply \*(C`skip\*(C' the complex parts and keep the rest.

<body> <!--[skip for="wml"]--> unecessary parts for the WML pages <!--[end_skip]--> useful parts for the WML pages </body>

The links reconstruction engine is \s-1IMHO\s0 the most important part of Html2Wml, because it's this engine that allows you to reconstruct the links of the \s-1HTML\s0 document being converted. It has two modes, depending upon whether Html2Wml was launched from the shell or as a \s-1CGI\s0.

When used as a \s-1CGI\s0, this engine will reconstructs the links of the \s-1HTML\s0 document so that all the urls will be passed to Html2Wml in order to convert the pointed files (pages or images). This is completly automatic and can't be customized for now (but I don't think it would be really useful).

When used from the shell, this engine reconstructs the links with the given templates. Note that absolute URLs will be left untouched. The templates can be customized using the following syntax. This template controls the reconstruction of the \*(C`href\*(C' attribute of the \*(C`A\*(C' tag. Its value can be changed using the \*(--hreftmpl option. Default value is \*(C`"{FILEPATH}{FILENAME}{$FILETYPE =~ s/s?html?/wml/o; $FILETYPE}"\*(C'. This template controls the reconstruction of the \*(C`src\*(C' attribute of the \*(C`IMG\*(C' tag. Its value can be changed using the \*(--srctmpl option. Default value is \*(C`"{FILEPATH}{FILENAME}{$FILETYPE =~ s/gif|png|jpe?g/wbmp/o; $FILETYPE}"\*(C' The template is a string that contains the new \s-1URL\s0. More precisely, it's a Text::Template template. Parameters can be interpolated as a constant or as a variable. The template is embraced between curcly bracets, and can contain any valid Perl code.

The simplest form of a template is \*(C`{PARAM}\*(C' which just returns the value of \s-1PARAM\s0. If you want to do something more complex, you can use the corresponding variable; for example \*(C`{"foo $PARAM bar"}\*(C', or \*(C`{join "_", split " ", PARAM}\*(C'.

You may read the Text::Template manpage for more information on what is possible within a template.

If the original \s-1URL\s0 contained a query part or a fragment part, then they will be appended to the result of the template. This parameter contains the original \s-1URL\s0 from the \*(C`href\*(C' or \*(C`src\*(C' attribute. This parameter contains the base name of the file. This parameter contains the leading path of the file. This parameter contains the suffix of the file.

This can be resumed this way:

URL = http://www.server.net/path/to/my/page.html ------------^^^^ ---- | | \ | | \ FILEPATH FILENAME FILETYPE Note that \*(C`FILETYPE\*(C' contains all the extensions of the file, so if its name is index.html.fr for example, \*(C`FILETYPE\*(C' contains "\*(C`.html.fr\*(C'". To add a path option:

{URL}$wap Using Apache, you can then add a Rewrite directive so that \s-1URL\s0 ending with \*(C`$wap\*(C' will be redirected to Html2Wml:

RewriteRule ^(/.*)\$wap$ /cgi-bin/html2wml.cgi?url=$1 To change the extension of an image:

{FILEPATH}{FILENAME}.wbmp

Html2Wml uses \s-1LWP\s0 built-in proxy support. It is activated by default, and loads the proxy settings from the environment variables, using the same variables as many others programs. Each protocol (http, ftp, etc) can be mapped to use a proxy server by setting a variable of the form \*(C`PROTOCOL_proxy\*(C'. Example: use \*(C`http_proxy\*(C' to define the proxy for http access, \*(C`ftp_proxy\*(C' for ftp access. In the shell, this is only a matter of defining the variable.

For Bourne shell:

$ export http_proxy="http://proxy.domain.com:8080/" For C-shell:

% setenv http_proxy "http://proxy.domain.com:8080/" Under Apache, you can add this directive to your configuration file:

SetEnv http_proxy "http://proxy.domain.com:8080" but this has the default that another \s-1CGI\s0, or another program, can use this to access external ressources. A better way is to edit Html2Wml and fill the option \*(C`proxy-server\*(C' with the appropriate value.

Html2Wml tries to make correct \s-1WML\s0 documents, but the well-formedness and the validity of the document are not guarantied.

Inverted tags (like "<b>bold <i>italic</b></i>") may produce unexpected results. But only bad softwares do bad stuff like this.

This is the web site of the Html2Wml project, hosted by SourceForge.net. All the stable releases can be downloaded from this site. [ http://www.html2wml.org/ ] This is the web site of the author, where you can find the archives of all the releases of Html2Wml. [ http://www.maddingue.org/softwares/ ] This is the official site of the \s-1WAP\s0 Forum. You can find some technical information, as the specifications of all the technologies associated with the \s-1WAP\s0. [ http://www.wapforum.org/ ] This site has some useful information and links. In particular, it has a quite well done \s-1FAQ\s0. [ http://www.wap.com/ ] Altough not directly related to the Wap stuff, you may find useful to read the specifications of the \s-1XML\s0 (\s-1WML\s0 is an \s-1XML\s0 application), and the specifications of the different stylesheet languages (\s-1CSS\s0 and \s-1XSL\s0), which include support for low-resolution devices. [ http://www.w3.org/ ] This web site is dedicated to Mobile UniX systems. It leads you to a lot of useful hands-on information about installing and running Linux and \s-1BSD\s0 on laptops, PDAs and other mobile computer devices. [ http://www.tuxmobil.org/ ] This is a very handful utility which corrects your \s-1HTML\s0 files so that they conform to W3C standards. [ http://www.w3.org/People/Raggett/tidy ] Kannel is an open source Wap and \s-1SMS\s0 gateway. A \s-1WML\s0 compiler is included in the distribution. [ http://www.kannel.org/ ] This is a collection of utilities for \s-1WML\s0 programmers. This include a compiler, a decompiler, a viewer and a \s-1WBMP\s0 converter. [ http://pwot.co.uk/wml/ ] Opera is originaly a Web browser, but the version 5 has a good support for \s-1XML\s0 and \s-1WML\s0. Opera is available for free for several systems. [ http://www.opera.com/ ] wApua is an open source \s-1WML\s0 browser written in Perl/Tk. It's easy to intall and to use. Its support for \s-1WML\s0 is incomplete, but sufficient for testing purpose. [ http://fsinfo.cs.uni-sb.de/~abe/wApua/ ] Tofoa is an open source Wap emulator written in Python. Its installation is quite difficult, and its incomplete \s-1WML\s0 support makes it produce strange results, even with valid \s-1WML\s0 documents. [ http://tofoa.free-system.com/ ] EzWAP, from \s-1EZOS\s0, is a commercial \s-1WML\s0 browser freely available for Windows 9x, \s-1NT\s0, 2000 and \s-1CE\s0. Compared to others Windows \s-1WML\s0 browsers, it requires very few resources, and is quite stable. Its support for the \s-1WML\s0 specs seems quite complete. A very good software. [ http://www.ezos.com/ ] Deck-It is a commercial Wap phone emulator, available for Windows and Linux/Intel only. It's a very good piece of software which really show how \s-1WML\s0 pages are rendered on a Wap phone, but one of its major default is that it cannot read local files. [ http://www.pyweb.com/tools/ ] Klondike \s-1WAP\s0 Browser is a commercial \s-1WAP\s0 browser available for Windows and PocketPC. [ http://www.apachesoftware.com/ ] WinWAP is a commercial Wap browser, freely available for Windows. [ http://www.winwap.org/ ] WAPman from EdgeMatrix, is a commercial \s-1WAP\s0 browser available for Windows and PalmOS. [ http://www.edgematrix.com/edge/control/MainContentBean?page=downloads ] Wireless Companion, from YourWap.com, is a \s-1WAP\s0 emulator available for Windows. [ http://www.yourwap.com/ ] Mobilizer is a Wap emulator available for Windows and Unix. [ http://mobilizer.sourceforge.net/ ] QWmlBrowser (formerly known as \s-1WML\s0 BRowser) is an open source \s-1WML\s0 browser, written using the Qt toolkit. [ http://www.wmlbrowser.org/ ] Wapsody, developed by \s-1IBM\s0, is a freely available simulation environment that implements the \s-1WAP\s0 specification. It also features a \s-1WML\s0 browser which can be run stand-alone. As Wapsody is written in Java/Swing, it should work on any system. [ http://alphaworks.ibm.com/aw.nsf/techmain/wapsody ] WAPreview is a Wap emulator written in Java. As it uses an \s-1HTML\s0 based \s-1UI\s0 and needs a local web proxy, it runs quite slowly. [ http://wapreview.sourceforge.net ] PicoWap is a small \s-1WML\s0 browser made by three French students. [ http://membres.lycos.fr/picowap/ ]

Werner Heuser, for his numerous ideas, advices and his help for the debugging

Igor Khristophorov, for his numerous suggestions and patches

And all the people that send me bug reports: Daniele Frijia, Axel Jerabek, Ouyang

Se\*'bastien Aperghis-Tramoni <[email protected]<gt>

Copyright (C)2000, 2001, 2002 Se\*'bastien Aperghis-Tramoni

This program is free software. You can redistribute it and/or modify it under the terms of the \s-1GNU\s0 General Public License, version 2 or later.