Packaging Plugins for Plugin Central with pjo.py

Authors: Ollie Rutherfurd
Alan Ezust
Contact: jedit-devel@lists.sourceforge.net
Date: 2007-10-06

Since you are reading this document, you are probably interested in plugin packaging. This must mean you have written a plugin, or fixed bugs in one, and want to release it yourself.

First, since you are going to make very frequent use of the sourceforge.net website, and it is notorious for embedding a lot of extra javascript that does nothing but invade your privacy, flash ads at you, and in general, slow down your browser in a most obnoxious way, it is highly recommended that you install the firefox extension, mozilla-noscript (www.noscript.net), allow javascript by sourceforge.net, and deny from doubleclick.net or google-analytics.com. Problem solved, and sourceforge is nice and fast again.

In order to gain access to the tools and files that allow you to release plugins to the general public, you must join the sourceforge project, jedit-plugins.

Contact one of the administrators of this project to get assistance during your first plugin release. The admin will give you access to the tools and webpages that we use when we release plugins.

pjo.py is a Python script for packaging plugins for jEdit Plugin Central. This document is a guide to using this script.

Packaging plugins can be a time-consuming and tedious task. One must get get the source and any dependencies, review the code, compile against the specified versions of jEdit and plugins, create packages, and perform upload them three times to three different places on sf.net. pjo.py is a Python script which cuts down on some of the grunt work. It can checkout plugins from CVS or SVN, fetch dependencies, create packages, verify JAR file integrity, ensure that required jars are included in the created packages, and verify package downloads.

In addition, this project serves as the documentation and scripts repository for anything else related to packaging and releasing plugins. Consider it always under development.

Note

pjo.py works best with plugins that are consistently packaged. For example, when the name of the plugin differs from the directory in CVS, it gets all out of joint.

Contents

Configuration

Variables

DOWNLOAD
Directory to download plugins and version of jEdit to. May be specified using --download-dir parameter, or with DOWNLOAD in your ".pjorc".
INSTALL
Where jEdit versions are installed.
MIRRORS
Space delimited sourceforge.net download mirrors. They are tried in order. For example, aleron unc umn.
SETTINGS
Directory containing jEdit settings directories. Individual settings directories will be named .version, within this directory. For example, .4.2pre7.
TIMEOUT
Server timeout, used when downloading from sf.net.
USERNAME
Sourceforge.net username. This is used for cvs/svn checkouts and uploading packages. If not specified, pjo will user your local username.

.pjorc

When run, pjo looks for a file named .pjorc in your home directory and the current directory, and if found "executes" them in that order. A .pjorc in your home directory is a good way to keep default settings.

Here's an example:

# where to find jEdit settings, install plugins, etc..
# for example, C:/jEdit/.4.2pre13
export SETTINGS=C:/jEdit/

# where jEdit versions are installed
export INSTALL=C:/jEdit/

# where to put/look for downloaded files
export DOWNLOAD=C:/sandbox/plugins.jedit.org/

# download mirrors, tried in order
export MIRRORS=aleron unc umn

# socket timeout when downloading files
export TIMEOUT=30

# by if not specified, your current username
# is picked up from the system
export USERNAME=orutherfurd

# $Id: sample_pjorc 6528 2006-03-09 17:19:37Z ezust $
# :mode=properties:

Note that any command-line arguments given to pjo override anything in a .pjorc file.

Directory Structure

pjo works with the following directory structure, where each plugin is in its own directory, named <plugin-name>-<version>, however, pjo will create the plugin directories using the checkout command, see Downloading Plugin Source Code.

FindFile-1.0/
  FindFile/
    build.xml
    ...
  FindFile.jar
HeadlinePlugin-1.1.3/
  HeadlinePlugin/
    build.xml

Downloading and installing jEdit versions

Each plugin needs to be tested against a specific version of jEdit, according to its dependencies. To download and install a specific version of jEdit for pjo to use:

./pjo.py -c download 4.3pre6
./pjo.py -c install 4.3pre6

This will download and install jedit 4.3pre6 in the DOWNLOAD_DIR indicated by your .pjorc file. Note that the settings directories for each version of jEdit will be in teh same location, in a directory called .4.3pre6 or .4.2.

Downloading Plugin Source Code

checkout (or co) checks out the source code for a plugin release from CVS. checkout creates a <name>-<version> directory.

Usage:

checkout <name>-<version> <tag>

Example:

$ pjo.py -c co FindFile-1.0 rel-1-0

To checkout plugins from Subversion, use command svnco instead. Everything else is the same:

$ pjo.py -c svnco Highlight-1.4.1 V_1_4_1

Installing Plugins

install (or in) can be used to install plugins required to build a plugin or a version of jEdit.

Usage:

install <name>-<version> [<name>-<version>]+ <version>

Example (installing a plugin):

pjo.py -c install ErrorList-1.3.2 InfoViewer-1.2 4.2pre7

Example (installing jedit 4.2 pre7 (the editor):

pjo.py -c install 4.2pre7

For install to work, you must have defined INSTALL, SETTINGS, MIRRORS, and DOWNLOAD.

Compiling

It's easiest when plugin authors use properties jedit.install.dir and install.dir, and targets clean and dist exist. Then one can do:

ant -Djedit.install.dir=C:/jEdit/4.2pre11 \
    -Ddocbook.xsl=C:/Share/docbook-xsl-1.65.1 \
    -Dinstall.dir=c:/.4.2pre11/jars \
    clean dist clean

However, often these properties are named something else. Also, if settings are not respectively ../.. and .., those should be set as the default.

In general, if you are packaging a plugin for someone else and the build.xml file is old and doesn't work properly, feel free to re-write it as a build.xml file that takes advantage of the build-support plugin project (on the CVS server as plugins/build-support).

Packaging

package (or pkg) is one of pjo's most useful commands. It creates packages for a plugin. For pkg to package a plugin, the plugin's jar file and all included jar files must be located in the <name>-<version> directory -- on the same level as the plugin's source code.

Usage: pkg <name>-<version>

Example:

2003-03-19$ pjo.py -c pkg JakartaCommons-0.4.2
Packaging JakartaCommons 0.4.2
Verifying jars... verifying JakartaCommons-0.4.2\bcel.jar
verifying JakartaCommons-0.4.2\commons-collections.jar
verifying JakartaCommons-0.4.2\commons-httpclient-2.0-rc2.jar
verifying JakartaCommons-0.4.2\commons-logging.jar
verifying JakartaCommons-0.4.2\JakartaCommons.jar
loading properties...
WARNING: no `docs` property
verifying JakartaCommons-0.4.2\log4j.jar
OK
Verifying properties... OK
creating JakartaCommons-0.4.2-bin.zip ... OK
creating JakartaCommons-0.4.2.zip ... OK
creating JakartaCommons-0.4.2-bin.tgz ... OK
creating JakartaCommons-0.4.2.tgz ... OK
creating CHECKSUMS... OK

Help

To see all of pjo's options, run pjo.py --help:

usage: pjo.py [options] [directory]+

options:
  -h, --help            show this help message and exit
  -b <date>, --batch=<date>
                        Batch date (YYYY-MM-DD)
  -c COMMANDS, --command=COMMANDS
                        command to execute
  -i <dir>, --install-dir=<dir>
                        directory for jEdit installations
  -d <dir>, --download-dir=<dir>
                        directory for plugin and jEdit downloads
  -r <path>, --pjorc=<path>
                        pjorc file to run after defaults
  -s <dir>, --settings-dir=<dir>
                        directory containing .jedit directories
  -t <num>, --timeout=<num>
                        socket timeout, in seconds
  -u USERNAME, --username=USERNAME
                        sf.net username
  --debug               enable tracebacks in errors

To get help on commands, use the help command:

Usage: help [command].

Examples:

$pjo.py -c help

Documented commands (type help <topic>):
========================================
EOF       download  exit    install  patch  upload
checkout  svnco     env     export  package  props  verify

Undocumented commands:
======================
co  dl  help  in  pkg  up

To get help for a specify command, do:

$ pjo.py -c help checkout
get source, by tag, for a plugin from CVS (for preparing release)
Usage: checkout directory tag

jv.py

jv.py is a Python script that launches a version of jEdit with a specific settings directory for that version. If the desired version of jEdit is not installed, it will attempt to download and install it before running it.

Example:

jv.py 4.2pre11

Note

If installing on Windows, the installation of jEditLauncher will throw up a couple of dialogs that you must respond to.

Tips

If you have to modify a build.xml file, send a patch to the author of the plugin. This way the next person to package the plugin may not have to do the same thing.

On Windows (at least w/my version cvs) sometimes files that have Windows line endings (rn) end up getting double. I assume cvs is converting from Unix to Windows line endings which double-spaces files. This can screw up things like property files where property values span multiple lines.

Upload Step 2

Now the packaged files need to be uploaded to the "incoming" directory on sf.net. This is in preparation for releasing the files.

From plugins.jedit.org, in the batch directory you uploaded your packaged files to, ncftp to upload.sf.net (as anonymous), cd to "incoming", and upload all package files.

Example:

/home/ezust/workspace/jedit/pjo/Highlight-1.4.1> ncftp ftp://upload.sf.net/incoming
Login successful.
Logged in to upload.sf.net.
Current remote directory is /incoming.
ncftp /incoming > mput *.tgz *.zip

Once they're uploaded using the sf.net project site, you can create the releases, and assign the files for the releases.

Upload Step 2 - Submitting files

Now we are ready to submit the files to sf.net and all of its mirrors, which involves filling out a web form on the sf.net jedit-plugins File Releases.

As you create releases in the web form, you select checkboxes next to files listed in the uploads directory, and the sf.net file release system sucks them up and distributes them to all the mirrors.

PLUGIN MANAGER DATABASE

After submitting files to sf.net, you need to update and regenerate the plugin manager xml plugin list with knowledge of this new release.

This is the data which is used to generate a plugin list.

Each plugin defines one or more branches. If the plugin has split into a 4.2final adn a 4.3preX version, then you might find the last final marked as "main" while the latest development version is marked as "devel". In other cases, we may see branches which are marked by version number. The key is, each branch can be displayed on PluginManager and be offered to jedit users running under different released versions.

Go to wiz and update the database, adding entries/dependency info for the plugin you just released. After you are finished, you run this script:

/home/groups/j/je/jedit-plugins/bin/generate-plugin-list

This generates the plugin list that gets downloaded whenever you run the jedit plugin manager. The information that is in the database determines which plugins are seen by jedit plugin manager and which are not.