path: root/buildbot/docs/buildbot.info-2

This is buildbot.info, produced by makeinfo version 4.11 from
buildbot.texinfo.

This is the BuildBot manual.

   Copyright (C) 2005,2006 Brian Warner

   Copying and distribution of this file, with or without
modification, are permitted in any medium without royalty provided
the copyright notice and this notice are preserved.


File: buildbot.info,  Node: Buildbot Web Resources,  Next: XMLRPC server,  Prev: WebStatus Configuration Parameters,  Up: WebStatus

7.1.2 Buildbot Web Resources
----------------------------

Certain URLs are "magic", and the pages they serve are created by
code in various classes in the `buildbot.status.web' package instead
of being read from disk. The most common way to access these pages is
for the buildmaster admin to write or modify the `index.html' page to
contain links to them. Of course other project web pages can contain
links to these buildbot pages as well.

   Many pages can be modified by adding query arguments to the URL.
For example, a page which shows the results of the most recent build
normally does this for all builders at once. But by appending
"?builder=i386" to the end of the URL, the page will show only the
results for the "i386" builder. When used in this way, you can add
multiple "builder=" arguments to see multiple builders. Remembering
that URL query arguments are separated _from each other_ with
ampersands, a URL that ends in "?builder=i386&builder=ppc" would show
builds for just those two Builders.

   The `branch=' query argument can be used on some pages. This
filters the information displayed by that page down to only the builds
or changes which involved the given branch. Use `branch=trunk' to
reference the trunk: if you aren't intentionally using branches,
you're probably using trunk. Multiple `branch=' arguments can be used
to examine multiple branches at once (so appending
`?branch=foo&branch=bar' to the URL will show builds involving either
branch). No `branch=' arguments means to show builds and changes for
all branches.

   Some pages may include the Builder name or the build number in the
main part of the URL itself. For example, a page that describes Build
#7 of the "i386" builder would live at `/builders/i386/builds/7'.

   The table below lists all of the internal pages and the URLs that
can be used to access them.

   NOTE: of the pages described here, `/slave_status_timeline' and
`/last_build' have not yet been implemented, and `/xmlrpc' has only a
few methods so far. Future releases will improve this.

`/waterfall'
     This provides a chronologically-oriented display of the activity
     of all builders. It is the same display used by the Waterfall
     display.

     By adding one or more "builder=" query arguments, the Waterfall
     is restricted to only showing information about the given
     Builders. By adding one or more "branch=" query arguments, the
     display is restricted to showing information about the given
     branches. In addition, adding one or more "category=" query
     arguments to the URL will limit the display to Builders that
     were defined with one of the given categories.

     A 'show_events=true' query argument causes the display to include
     non-Build events, like slaves attaching and detaching, as well as
     reconfiguration events. 'show_events=false' hides these events.
     The default is to show them.

     The `last_time=', `first_time=', and `show_time=' arguments will
     control what interval of time is displayed. The default is to
     show the latest events, but these can be used to look at earlier
     periods in history. The `num_events=' argument also provides a
     limit on the size of the displayed page.

     The Waterfall has references to resources many of the other
     portions of the URL space: `/builders' for access to individual
     builds, `/changes' for access to information about source code
     changes, etc.

`/rss'
     This provides a rss feed summarizing all failed builds. The same
     query-arguments used by 'waterfall' can be added to filter the
     feed output.

`/atom'
     This provides an atom feed summarizing all failed builds. The
     same query-arguments used by 'waterfall' can be added to filter
     the feed output.

`/builders/$BUILDERNAME'
     This describes the given Builder, and provides buttons to force
     a build.

`/builders/$BUILDERNAME/builds/$BUILDNUM'
     This describes a specific Build.

`/builders/$BUILDERNAME/builds/$BUILDNUM/steps/$STEPNAME'
     This describes a specific BuildStep.

`/builders/$BUILDERNAME/builds/$BUILDNUM/steps/$STEPNAME/logs/$LOGNAME'
     This provides an HTML representation of a specific logfile.

`/builders/$BUILDERNAME/builds/$BUILDNUM/steps/$STEPNAME/logs/$LOGNAME/text'
     This returns the logfile as plain text, without any HTML coloring
     markup. It also removes the "headers", which are the lines that
     describe what command was run and what the environment variable
     settings were like. This maybe be useful for saving to disk and
     feeding to tools like 'grep'.

`/changes'
     This provides a brief description of the ChangeSource in use
     (*note Change Sources::).

`/changes/NN'
     This shows detailed information about the numbered Change: who
     was the author, what files were changed, what revision number
     was represented, etc.

`/buildslaves'
     This summarizes each BuildSlave, including which Builders are
     configured to use it, whether the buildslave is currently
     connected or not, and host information retrieved from the
     buildslave itself.

`/one_line_per_build'
     This page shows one line of text for each build, merging
     information from all Builders(1). Each line specifies the name
     of the Builder, the number of the Build, what revision it used,
     and a summary of the results. Successful builds are in green,
     while failing builds are in red. The date and time of the build
     are added to the right-hand edge of the line. The lines are
     ordered by build finish timestamp.

     One or more `builder=' or `branch=' arguments can be used to
     restrict the list. In addition, a `numbuilds=' argument will
     control how many lines are displayed (20 by default).

`/one_box_per_builder'
     This page shows a small table, with one box for each Builder,
     containing the results of the most recent Build. It does not
     show the individual steps, or the current status. This is a
     simple summary of buildbot status: if this page is green, then
     all tests are passing.

     As with `/one_line_per_build', this page will also honor
     `builder=' and `branch=' arguments.

`/about'
     This page gives a brief summary of the Buildbot itself: software
     version, versions of some libraries that the Buildbot depends
     upon, etc. It also contains a link to the buildbot.net home page.

`/slave_status_timeline'
     (note: this page has not yet been implemented)

     This provides a chronological display of configuration and
     operational events: master startup/shutdown, slave
     connect/disconnect, and config-file changes. When a config-file
     reload is abandoned because of an error in the config file, the
     error is displayed on this page.

     This page does not show any builds.

`/last_build/$BUILDERNAME/status.png'
     This returns a PNG image that describes the results of the most
     recent build, which can be referenced in an IMG tag by other
     pages, perhaps from a completely different site. Use it as you
     would a webcounter.


   There are also a set of web-status resources that are intended for
use by other programs, rather than humans.

`/xmlrpc'
     This runs an XML-RPC server which can be used to query status
     information about various builds. See *note XMLRPC server:: for
     more details.


   ---------- Footnotes ----------

   (1) Apparently this is the same way http://buildd.debian.org
displays build status


File: buildbot.info,  Node: XMLRPC server,  Next: HTML Waterfall,  Prev: Buildbot Web Resources,  Up: WebStatus

7.1.3 XMLRPC server
-------------------

When using WebStatus, the buildbot runs an XML-RPC server at
`/xmlrpc' that can be used by other programs to query build status.
The following table lists the methods that can be invoked using this
interface.

`getAllBuildsInInterval(start, stop)'
     Return a list of builds that have completed after the 'start'
     timestamp and before the 'stop' timestamp. This looks at all
     Builders.

     The timestamps are integers, interpreted as standard unix
     timestamps (seconds since epoch).

     Each Build is returned as a tuple in the form: `(buildername,
     buildnumber, build_end, branchname, revision, results, text)'

     The buildnumber is an integer. 'build_end' is an integer (seconds
     since epoch) specifying when the build finished.

     The branchname is a string, which may be an empty string to
     indicate None (i.e. the default branch). The revision is a
     string whose meaning is specific to the VC system in use, and
     comes from the 'got_revision' build property. The results are
     expressed as a string, one of ('success', 'warnings', 'failure',
     'exception'). The text is a list of short strings that ought to
     be joined by spaces and include slightly more data about the
     results of the build.

`getBuild(builder_name, build_number)'
     Return information about a specific build.

     This returns a dictionary (aka "struct" in XMLRPC terms) with
     complete information about the build. It does not include the
     contents of the log files, but it has just about everything else.



File: buildbot.info,  Node: HTML Waterfall,  Prev: XMLRPC server,  Up: WebStatus

7.1.4 HTML Waterfall
--------------------

The `Waterfall' status target, deprecated as of 0.7.6, is a subset of
the regular `WebStatus' resource (*note WebStatus::).  This section
(and the `Waterfall' class itself) will be removed from a future
release.

     from buildbot.status import html
     w = html.WebStatus(http_port=8080)
     c['status'].append(w)


File: buildbot.info,  Node: MailNotifier,  Next: IRC Bot,  Prev: WebStatus,  Up: Status Delivery

7.2 MailNotifier
================

The buildbot can also send email when builds finish. The most common
use of this is to tell developers when their change has caused the
build to fail. It is also quite common to send a message to a mailing
list (usually named "builds" or similar) about every build.

   The `MailNotifier' status target is used to accomplish this. You
configure it by specifying who mail should be sent to, under what
circumstances mail should be sent, and how to deliver the mail. It can
be configured to only send out mail for certain builders, and only
send messages when the build fails, or when the builder transitions
from success to failure. It can also be configured to include various
build logs in each message.

   By default, the message will be sent to the Interested Users list
(*note Doing Things With Users::), which includes all developers who
made changes in the build. You can add additional recipients with the
extraRecipients argument.

   Each MailNotifier sends mail to a single set of recipients. To send
different kinds of mail to different recipients, use multiple
MailNotifiers.

   The following simple example will send an email upon the
completion of each build, to just those developers whose Changes were
included in the build. The email contains a description of the Build,
its results, and URLs where more information can be obtained.

     from buildbot.status.mail import MailNotifier
     mn = MailNotifier(fromaddr="buildbot@example.org", lookup="example.org")
     c['status'].append(mn)

   To get a simple one-message-per-build (say, for a mailing list),
use the following form instead. This form does not send mail to
individual developers (and thus does not need the `lookup=' argument,
explained below), instead it only ever sends mail to the "extra
recipients" named in the arguments:

     mn = MailNotifier(fromaddr="buildbot@example.org",
                       sendToInterestedUsers=False,
                       extraRecipients=['listaddr@example.org'])

   In some cases it is desirable to have different information then
what is provided in a standard MailNotifier message. For this purpose
MailNotifier provides the argument customMesg (a function) which
allows for the creation of messages with unique content.

   For example it can be useful to display the last few lines of a
log file and recent changes when a builder fails:

     def message(attrs):
         logLines = 10
         text = list()
         text.append("STATUS: %s" % attrs['result'].title())
         text.append("")
         text.extend([c.asText() for c in attrs['changes']])
         text.append("")
         name, url, lines = attrs['logs'][-1]
         text.append("Last %d lines of '%s':" % (logLines, name))
         text.extend(["\t%s\n" % line for line in lines[len(lines)-logLines:]])
         text.append("")
         text.append("-buildbot")
         return ("\n".join(text), 'plain')

     mn = MailNotifier(fromaddr="buildbot@example.org",
                       sendToInterestedUsers=False,
                       mode='problem',
                       extraRecipients=['listaddr@example.org'],
                       customMesg=message)

   A customMesg function takes a single dict argument (see below) and
returns a tuple of strings. The first string is the complete text of
the message and the second is the message type ('plain' or 'html').
The 'html' type should be used when generating an HTML message:

     def message(attrs):
         logLines = 10
         text = list()
         text.append('<h4>Build status %s.</h4>' % (attrs['result'].title()))
         if attrs['changes']:
             text.append('<h4>Recent Changes:</h4>')
             text.extend([c.asHTML() for c in attrs['changes']])
         name, url, lines = attrs['logs'][-1]
         text.append('<h4>Last %d lines of "%s":</h4>' % (logLines, name))
         text.append('<p>')
         text.append('<br>'.join([line for line in lines[len(lines)-logLines:]]))
         text.append('</p>')
         text.append('<br><br>')
         text.append('Full log at: %s' % url)
         text.append('<br><br>')
         text.append('<b>-buildbot</b>')
         return ('\n'.join(text), 'html')

MailNotifier arguments
======================

`fromaddr'
     The email address to be used in the 'From' header.

`sendToInterestedUsers'
     (boolean). If True (the default), send mail to all of the
     Interested Users. If False, only send mail to the
     extraRecipients list.

`extraRecipients'
     (tuple of strings). A list of email addresses to which messages
     should be sent (in addition to the InterestedUsers list, which
     includes any developers who made Changes that went into this
     build). It is a good idea to create a small mailing list and
     deliver to that, then let subscribers come and go as they please.

`subject'
     (string). A string to be used as the subject line of the message.
     `%(builder)s' will be replaced with the name of the builder which
     provoked the message.

`mode'
     (string). Default to 'all'. One of:
    `all'
          Send mail about all builds, bothpassing and failing

    `failing'
          Only send mail about builds which fail

    `problem'
          Only send mail about a build which failed when the previous
          build has passed.  If your builds usually pass, then this
          will only send mail when a problem occurs.

`builders'
     (list of strings). A list of builder names for which mail should
     be sent. Defaults to None (send mail for all builds). Use either
     builders or categories, but not both.

`categories'
     (list of strings). A list of category names to serve status
     information for. Defaults to None (all categories). Use either
     builders or categories, but not both.

`addLogs'
     (boolean). If True, include all build logs as attachments to the
     messages. These can be quite large. This can also be set to a
     list of log names, to send a subset of the logs. Defaults to
     False.

`relayhost'
     (string). The host to which the outbound SMTP connection should
     be made. Defaults to 'localhost'

`lookup'
     (implementor of `IEmailLookup'). Object which provides
     IEmailLookup, which is responsible for mapping User names (which
     come from the VC system) into valid email addresses. If not
     provided, the notifier will only be able to send mail to the
     addresses in the extraRecipients list. Most of the time you can
     use a simple Domain instance. As a shortcut, you can pass as
     string: this will be treated as if you had provided Domain(str).
     For example, lookup='twistedmatrix.com' will allow mail to be
     sent to all developers whose SVN usernames match their
     twistedmatrix.com account names. See buildbot/status/mail.py for
     more details.

`customMesg'
     This is a optional function that can be used to generate a
     custom mail message. The customMesg function takes a single dict
     and must return a tuple containing the message text and type
     ('html' or 'plain'). Below is a list of availale keys in the
     dict passed to customMesg:

    `builderName'
          (str) Name of the builder that generated this event.

    `projectName'
          (str) Name of the project.

    `mode'
          (str) Mode set in MailNotifier. (failing, passing, problem).

    `result'
          (str) Builder result as a string. 'success', 'warnings',
          'failure', 'skipped', or 'exception'

    `buildURL'
          (str) URL to build page.

    `buildbotURL'
          (str) URL to buildbot main page.

    `buildText'
          (str) Build text from build.getText().

    `slavename'
          (str) Slavename.

    `reason'
          (str) Build reason from build.getReason().

    `responsibleUsers'
          (List of str) List of responsible users.

    `branch'
          (str) Name of branch used. If no SourceStamp exists branch
          is an empty string.

    `revision'
          (str) Name of revision used. If no SourceStamp exists
          revision is an empty string.

    `patch'
          (str) Name of patch used. If no SourceStamp exists patch is
          an empty string.

    `changes'
          (list of objs) List of change objects from SourceStamp. A
          change object has the following useful information:
         `who'
               (str) who made this change

         `revision'
               (str) what VC revision is this change

         `branch'
               (str) on what branch did this change occur

         `when'
               (str) when did this change occur

         `files'
               (list of str) what files were affected in this change

         `comments'
               (str) comments reguarding the change.
          The functions asText and asHTML return a list of strings
          with the above information formatted.

    `logs'
          (List of Tuples) List of tuples where each tuple contains
          the log name, log url, and log contents as a list of
          strings.


File: buildbot.info,  Node: IRC Bot,  Next: PBListener,  Prev: MailNotifier,  Up: Status Delivery

7.3 IRC Bot
===========

The `buildbot.status.words.IRC' status target creates an IRC bot
which will attach to certain channels and be available for status
queries. It can also be asked to announce builds as they occur, or be
told to shut up.

     from buildbot.status import words
     irc = words.IRC("irc.example.org", "botnickname",
                     channels=["channel1", "channel2"],
                     password="mysecretpassword",
                     notify_events={
                       'exception': 1,
                       'successToFailure': 1,
                       'failureToSuccess': 1,
                     })
     c['status'].append(irc)

   Take a look at the docstring for `words.IRC' for more details on
configuring this service. The `password' argument, if provided, will
be sent to Nickserv to claim the nickname: some IRC servers will not
allow clients to send private messages until they have logged in with
a password.

   To use the service, you address messages at the buildbot, either
normally (`botnickname: status') or with private messages (`/msg
botnickname status'). The buildbot will respond in kind.

   Some of the commands currently available:

`list builders'
     Emit a list of all configured builders

`status BUILDER'
     Announce the status of a specific Builder: what it is doing
     right now.

`status all'
     Announce the status of all Builders

`watch BUILDER'
     If the given Builder is currently running, wait until the Build
     is finished and then announce the results.

`last BUILDER'
     Return the results of the last build to run on the given Builder.

`join CHANNEL'
     Join the given IRC channel

`leave CHANNEL'
     Leave the given IRC channel

`notify on|off|list EVENT'
     Report events relating to builds.  If the command is issued as a
     private message, then the report will be sent back as a private
     message to the user who issued the command.  Otherwise, the
     report will be sent to the channel.  Available events to be
     notified are:

    `started'
          A build has started

    `finished'
          A build has finished

    `success'
          A build finished successfully

    `failed'
          A build failed

    `exception'
          A build generated and exception

    `xToY'
          The previous build was x, but this one is Y, where x and Y
          are each one of success, warnings, failure, exception
          (except Y is capitalized).  For example: successToFailure
          will notify if the previous build was successful, but this
          one failed

`help COMMAND'
     Describe a command. Use `help commands' to get a list of known
     commands.

`source'
     Announce the URL of the Buildbot's home page.

`version'
     Announce the version of this Buildbot.

   Additionally, the config file may specify default notification
options as shown in the example earlier.

   If the `allowForce=True' option was used, some addtional commands
will be available:

`force build BUILDER REASON'
     Tell the given Builder to start a build of the latest code. The
     user requesting the build and REASON are recorded in the Build
     status. The buildbot will announce the build's status when it
     finishes.

`stop build BUILDER REASON'
     Terminate any running build in the given Builder. REASON will be
     added to the build status to explain why it was stopped. You
     might use this if you committed a bug, corrected it right away,
     and don't want to wait for the first build (which is destined to
     fail) to complete before starting the second (hopefully fixed)
     build.


File: buildbot.info,  Node: PBListener,  Next: Writing New Status Plugins,  Prev: IRC Bot,  Up: Status Delivery

7.4 PBListener
==============

     import buildbot.status.client
     pbl = buildbot.status.client.PBListener(port=int, user=str,
                                             passwd=str)
     c['status'].append(pbl)

   This sets up a PB listener on the given TCP port, to which a
PB-based status client can connect and retrieve status information.
`buildbot statusgui' (*note statusgui::) is an example of such a
status client. The `port' argument can also be a strports
specification string.


File: buildbot.info,  Node: Writing New Status Plugins,  Prev: PBListener,  Up: Status Delivery

7.5 Writing New Status Plugins
==============================

TODO: this needs a lot more examples

   Each status plugin is an object which provides the
`twisted.application.service.IService' interface, which creates a
tree of Services with the buildmaster at the top [not strictly true].
The status plugins are all children of an object which implements
`buildbot.interfaces.IStatus', the main status object. From this
object, the plugin can retrieve anything it wants about current and
past builds. It can also subscribe to hear about new and upcoming
builds.

   Status plugins which only react to human queries (like the
Waterfall display) never need to subscribe to anything: they are idle
until someone asks a question, then wake up and extract the
information they need to answer it, then they go back to sleep.
Plugins which need to act spontaneously when builds complete (like
the MailNotifier plugin) need to subscribe to hear about new builds.

   If the status plugin needs to run network services (like the HTTP
server used by the Waterfall plugin), they can be attached as Service
children of the plugin itself, using the `IServiceCollection'
interface.


File: buildbot.info,  Node: Command-line tool,  Next: Resources,  Prev: Status Delivery,  Up: Top

8 Command-line tool
*******************

The `buildbot' command-line tool can be used to start or stop a
buildmaster or buildbot, and to interact with a running buildmaster.
Some of its subcommands are intended for buildmaster admins, while
some are for developers who are editing the code that the buildbot is
monitoring.

* Menu:

* Administrator Tools::
* Developer Tools::
* Other Tools::
* .buildbot config directory::


File: buildbot.info,  Node: Administrator Tools,  Next: Developer Tools,  Prev: Command-line tool,  Up: Command-line tool

8.1 Administrator Tools
=======================

The following `buildbot' sub-commands are intended for buildmaster
administrators:

create-master
=============

This creates a new directory and populates it with files that allow it
to be used as a buildmaster's base directory.

     buildbot create-master BASEDIR

create-slave
============

This creates a new directory and populates it with files that let it
be used as a buildslave's base directory. You must provide several
arguments, which are used to create the initial `buildbot.tac' file.

     buildbot create-slave BASEDIR MASTERHOST:PORT SLAVENAME PASSWORD

start
=====

This starts a buildmaster or buildslave which was already created in
the given base directory. The daemon is launched in the background,
with events logged to a file named `twistd.log'.

     buildbot start BASEDIR

stop
====

This terminates the daemon (either buildmaster or buildslave) running
in the given directory.

     buildbot stop BASEDIR

sighup
======

This sends a SIGHUP to the buildmaster running in the given directory,
which causes it to re-read its `master.cfg' file.

     buildbot sighup BASEDIR


File: buildbot.info,  Node: Developer Tools,  Next: Other Tools,  Prev: Administrator Tools,  Up: Command-line tool

8.2 Developer Tools
===================

These tools are provided for use by the developers who are working on
the code that the buildbot is monitoring.

* Menu:

* statuslog::
* statusgui::
* try::


File: buildbot.info,  Node: statuslog,  Next: statusgui,  Prev: Developer Tools,  Up: Developer Tools

8.2.1 statuslog
---------------

     buildbot statuslog --master MASTERHOST:PORT

   This command starts a simple text-based status client, one which
just prints out a new line each time an event occurs on the
buildmaster.

   The `--master' option provides the location of the
`buildbot.status.client.PBListener' status port, used to deliver
build information to realtime status clients. The option is always in
the form of a string, with hostname and port number separated by a
colon (`HOSTNAME:PORTNUM'). Note that this port is _not_ the same as
the slaveport (although a future version may allow the same port
number to be used for both purposes). If you get an error message to
the effect of "Failure: twisted.cred.error.UnauthorizedLogin:", this
may indicate that you are connecting to the slaveport rather than a
`PBListener' port.

   The `--master' option can also be provided by the `masterstatus'
name in `.buildbot/options' (*note .buildbot config directory::).


File: buildbot.info,  Node: statusgui,  Next: try,  Prev: statuslog,  Up: Developer Tools

8.2.2 statusgui
---------------

If you have set up a PBListener (*note PBListener::), you will be able
to monitor your Buildbot using a simple Gtk+ application invoked with
the `buildbot statusgui' command:

     buildbot statusgui --master MASTERHOST:PORT

   This command starts a simple Gtk+-based status client, which
contains a few boxes for each Builder that change color as events
occur. It uses the same `--master' argument as the `buildbot
statuslog' command (*note statuslog::).


File: buildbot.info,  Node: try,  Prev: statusgui,  Up: Developer Tools

8.2.3 try
---------

This lets a developer to ask the question "What would happen if I
committed this patch right now?". It runs the unit test suite (across
multiple build platforms) on the developer's current code, allowing
them to make sure they will not break the tree when they finally
commit their changes.

   The `buildbot try' command is meant to be run from within a
developer's local tree, and starts by figuring out the base revision
of that tree (what revision was current the last time the tree was
updated), and a patch that can be applied to that revision of the tree
to make it match the developer's copy. This (revision, patch) pair is
then sent to the buildmaster, which runs a build with that
SourceStamp. If you want, the tool will emit status messages as the
builds run, and will not terminate until the first failure has been
detected (or the last success).

   There is an alternate form which accepts a pre-made patch file
(typically the output of a command like 'svn diff'). This "-diff"
form does not require a local tree to run from. See *Note try
--diff::.

   For this command to work, several pieces must be in place:

TryScheduler
============

The buildmaster must have a `scheduler.Try' instance in the config
file's `c['schedulers']' list. This lets the administrator control
who may initiate these "trial" builds, which branches are eligible
for trial builds, and which Builders should be used for them.

   The `TryScheduler' has various means to accept build requests: all
of them enforce more security than the usual buildmaster ports do.
Any source code being built can be used to compromise the buildslave
accounts, but in general that code must be checked out from the VC
repository first, so only people with commit privileges can get
control of the buildslaves. The usual force-build control channels can
waste buildslave time but do not allow arbitrary commands to be
executed by people who don't have those commit privileges. However,
the source code patch that is provided with the trial build does not
have to go through the VC system first, so it is important to make
sure these builds cannot be abused by a non-committer to acquire as
much control over the buildslaves as a committer has. Ideally, only
developers who have commit access to the VC repository would be able
to start trial builds, but unfortunately the buildmaster does not, in
general, have access to VC system's user list.

   As a result, the `TryScheduler' requires a bit more configuration.
There are currently two ways to set this up:

*jobdir (ssh)*
     This approach creates a command queue directory, called the
     "jobdir", in the buildmaster's working directory. The buildmaster
     admin sets the ownership and permissions of this directory to
     only grant write access to the desired set of developers, all of
     whom must have accounts on the machine. The `buildbot try'
     command creates a special file containing the source stamp
     information and drops it in the jobdir, just like a standard
     maildir. When the buildmaster notices the new file, it unpacks
     the information inside and starts the builds.

     The config file entries used by 'buildbot try' either specify a
     local queuedir (for which write and mv are used) or a remote one
     (using scp and ssh).

     The advantage of this scheme is that it is quite secure, the
     disadvantage is that it requires fiddling outside the buildmaster
     config (to set the permissions on the jobdir correctly). If the
     buildmaster machine happens to also house the VC repository,
     then it can be fairly easy to keep the VC userlist in sync with
     the trial-build userlist. If they are on different machines,
     this will be much more of a hassle. It may also involve granting
     developer accounts on a machine that would not otherwise require
     them.

     To implement this, the buildslave invokes 'ssh -l username host
     buildbot tryserver ARGS', passing the patch contents over stdin.
     The arguments must include the inlet directory and the revision
     information.

*user+password (PB)*
     In this approach, each developer gets a username/password pair,
     which are all listed in the buildmaster's configuration file.
     When the developer runs `buildbot try', their machine connects
     to the buildmaster via PB and authenticates themselves using
     that username and password, then sends a PB command to start the
     trial build.

     The advantage of this scheme is that the entire configuration is
     performed inside the buildmaster's config file. The
     disadvantages are that it is less secure (while the "cred"
     authentication system does not expose the password in plaintext
     over the wire, it does not offer most of the other security
     properties that SSH does). In addition, the buildmaster admin is
     responsible for maintaining the username/password list, adding
     and deleting entries as developers come and go.


   For example, to set up the "jobdir" style of trial build, using a
command queue directory of `MASTERDIR/jobdir' (and assuming that all
your project developers were members of the `developers' unix group),
you would first create that directory (with `mkdir MASTERDIR/jobdir
MASTERDIR/jobdir/new MASTERDIR/jobdir/cur MASTERDIR/jobdir/tmp; chgrp
developers MASTERDIR/jobdir MASTERDIR/jobdir/*; chmod g+rwx,o-rwx
MASTERDIR/jobdir MASTERDIR/jobdir/*'), and then use the following
scheduler in the buildmaster's config file:

     from buildbot.scheduler import Try_Jobdir
     s = Try_Jobdir("try1", ["full-linux", "full-netbsd", "full-OSX"],
                    jobdir="jobdir")
     c['schedulers'] = [s]

   Note that you must create the jobdir before telling the
buildmaster to use this configuration, otherwise you will get an
error. Also remember that the buildmaster must be able to read and
write to the jobdir as well. Be sure to watch the `twistd.log' file
(*note Logfiles::) as you start using the jobdir, to make sure the
buildmaster is happy with it.

   To use the username/password form of authentication, create a
`Try_Userpass' instance instead. It takes the same `builderNames'
argument as the `Try_Jobdir' form, but accepts an addtional `port'
argument (to specify the TCP port to listen on) and a `userpass' list
of username/password pairs to accept. Remember to use good passwords
for this: the security of the buildslave accounts depends upon it:

     from buildbot.scheduler import Try_Userpass
     s = Try_Userpass("try2", ["full-linux", "full-netbsd", "full-OSX"],
                      port=8031, userpass=[("alice","pw1"), ("bob", "pw2")] )
     c['schedulers'] = [s]

   Like most places in the buildbot, the `port' argument takes a
strports specification. See `twisted.application.strports' for
details.

locating the master
===================

The `try' command needs to be told how to connect to the
`TryScheduler', and must know which of the authentication approaches
described above is in use by the buildmaster. You specify the
approach by using `--connect=ssh' or `--connect=pb' (or `try_connect
= 'ssh'' or `try_connect = 'pb'' in `.buildbot/options').

   For the PB approach, the command must be given a `--master'
argument (in the form HOST:PORT) that points to TCP port that you
picked in the `Try_Userpass' scheduler. It also takes a `--username'
and `--passwd' pair of arguments that match one of the entries in the
buildmaster's `userpass' list. These arguments can also be provided
as `try_master', `try_username', and `try_password' entries in the
`.buildbot/options' file.

   For the SSH approach, the command must be given `--tryhost',
`--username', and optionally `--password' (TODO: really?) to get to
the buildmaster host. It must also be given `--trydir', which points
to the inlet directory configured above. The trydir can be relative
to the user's home directory, but most of the time you will use an
explicit path like `~buildbot/project/trydir'. These arguments can be
provided in `.buildbot/options' as `try_host', `try_username',
`try_password', and `try_dir'.

   In addition, the SSH approach needs to connect to a PBListener
status port, so it can retrieve and report the results of the build
(the PB approach uses the existing connection to retrieve status
information, so this step is not necessary). This requires a
`--master' argument, or a `masterstatus' entry in `.buildbot/options',
in the form of a HOSTNAME:PORT string.

choosing the Builders
=====================

A trial build is performed on multiple Builders at the same time, and
the developer gets to choose which Builders are used (limited to a set
selected by the buildmaster admin with the TryScheduler's
`builderNames=' argument). The set you choose will depend upon what
your goals are: if you are concerned about cross-platform
compatibility, you should use multiple Builders, one from each
platform of interest. You might use just one builder if that platform
has libraries or other facilities that allow better test coverage than
what you can accomplish on your own machine, or faster test runs.

   The set of Builders to use can be specified with multiple
`--builder' arguments on the command line. It can also be specified
with a single `try_builders' option in `.buildbot/options' that uses
a list of strings to specify all the Builder names:

     try_builders = ["full-OSX", "full-win32", "full-linux"]

specifying the VC system
========================

The `try' command also needs to know how to take the developer's
current tree and extract the (revision, patch) source-stamp pair.
Each VC system uses a different process, so you start by telling the
`try' command which VC system you are using, with an argument like
`--vc=cvs' or `--vc=tla'.  This can also be provided as `try_vc' in
`.buildbot/options'.

   The following names are recognized: `cvs' `svn' `baz' `tla' `hg'
`darcs'

finding the top of the tree
===========================

Some VC systems (notably CVS and SVN) track each directory
more-or-less independently, which means the `try' command needs to
move up to the top of the project tree before it will be able to
construct a proper full-tree patch. To accomplish this, the `try'
command will crawl up through the parent directories until it finds a
marker file. The default name for this marker file is
`.buildbot-top', so when you are using CVS or SVN you should `touch
.buildbot-top' from the top of your tree before running `buildbot
try'. Alternatively, you can use a filename like `ChangeLog' or
`README', since many projects put one of these files in their
top-most directory (and nowhere else). To set this filename, use
`--try-topfile=ChangeLog', or set it in the options file with
`try_topfile = 'ChangeLog''.

   You can also manually set the top of the tree with
`--try-topdir=~/trees/mytree', or `try_topdir = '~/trees/mytree''. If
you use `try_topdir', in a `.buildbot/options' file, you will need a
separate options file for each tree you use, so it may be more
convenient to use the `try_topfile' approach instead.

   Other VC systems which work on full projects instead of individual
directories (tla, baz, darcs, monotone, mercurial, git) do not require
`try' to know the top directory, so the `--try-topfile' and
`--try-topdir' arguments will be ignored.

   If the `try' command cannot find the top directory, it will abort
with an error message.

determining the branch name
===========================

Some VC systems record the branch information in a way that "try" can
locate it, in particular Arch (both `tla' and `baz'). For the others,
if you are using something other than the default branch, you will
have to tell the buildbot which branch your tree is using. You can do
this with either the `--branch' argument, or a `try_branch' entry in
the `.buildbot/options' file.

determining the revision and patch
==================================

Each VC system has a separate approach for determining the tree's base
revision and computing a patch.

`CVS'
     `try' pretends that the tree is up to date. It converts the
     current time into a `-D' time specification, uses it as the base
     revision, and computes the diff between the upstream tree as of
     that point in time versus the current contents. This works, more
     or less, but requires that the local clock be in reasonably good
     sync with the repository.

`SVN'
     `try' does a `svn status -u' to find the latest repository
     revision number (emitted on the last line in the "Status against
     revision: NN" message). It then performs an `svn diff -rNN' to
     find out how your tree differs from the repository version, and
     sends the resulting patch to the buildmaster. If your tree is not
     up to date, this will result in the "try" tree being created with
     the latest revision, then _backwards_ patches applied to bring it
     "back" to the version you actually checked out (plus your actual
     code changes), but this will still result in the correct tree
     being used for the build.

`baz'
     `try' does a `baz tree-id' to determine the fully-qualified
     version and patch identifier for the tree
     (ARCHIVE/VERSION-patch-NN), and uses the VERSION-patch-NN
     component as the base revision. It then does a `baz diff' to
     obtain the patch.

`tla'
     `try' does a `tla tree-version' to get the fully-qualified
     version identifier (ARCHIVE/VERSION), then takes the first line
     of `tla logs --reverse' to figure out the base revision. Then it
     does `tla changes --diffs' to obtain the patch.

`Darcs'
     `darcs changes --context' emits a text file that contains a list
     of all patches back to and including the last tag was made. This
     text file (plus the location of a repository that contains all
     these patches) is sufficient to re-create the tree. Therefore
     the contents of this "context" file _are_ the revision stamp for
     a Darcs-controlled source tree.

     So `try' does a `darcs changes --context' to determine what your
     tree's base revision is, and then does a `darcs diff -u' to
     compute the patch relative to that revision.

`Mercurial'
     `hg identify' emits a short revision ID (basically a truncated
     SHA1 hash of the current revision's contents), which is used as
     the base revision. `hg diff' then provides the patch relative to
     that revision. For `try' to work, your working directory must
     only have patches that are available from the same
     remotely-available repository that the build process'
     `step.Mercurial' will use.

`Git'
     `git branch -v' lists all the branches available in the local
     repository along with the revision ID it points to and a short
     summary of the last commit. The line containing the currently
     checked out branch begins with '* ' (star and space) while all
     the others start with '  ' (two spaces). `try' scans for this
     line and extracts the branch name and revision from it. Then it
     generates a diff against the base revision.


waiting for results
===================

If you provide the `--wait' option (or `try_wait = True' in
`.buildbot/options'), the `buildbot try' command will wait until your
changes have either been proven good or bad before exiting. Unless
you use the `--quiet' option (or `try_quiet=True'), it will emit a
progress message every 60 seconds until the builds have completed.

* Menu:

* try --diff::


File: buildbot.info,  Node: try --diff,  Prev: try,  Up: try

8.2.3.1 try -diff
.................

Sometimes you might have a patch from someone else that you want to
submit to the buildbot. For example, a user may have created a patch
to fix some specific bug and sent it to you by email. You've inspected
the patch and suspect that it might do the job (and have at least
confirmed that it doesn't do anything evil). Now you want to test it
out.

   One approach would be to check out a new local tree, apply the
patch, run your local tests, then use "buildbot try" to run the tests
on other platforms. An alternate approach is to use the `buildbot try
--diff' form to have the buildbot test the patch without using a
local tree.

   This form takes a `--diff' argument which points to a file that
contains the patch you want to apply. By default this patch will be
applied to the TRUNK revision, but if you give the optional
`--baserev' argument, a tree of the given revision will be used as a
starting point instead of TRUNK.

   You can also use `buildbot try --diff=-' to read the patch from
stdin.

   Each patch has a "patchlevel" associated with it. This indicates
the number of slashes (and preceding pathnames) that should be
stripped before applying the diff. This exactly corresponds to the
`-p' or `--strip' argument to the `patch' utility. By default
`buildbot try --diff' uses a patchlevel of 0, but you can override
this with the `-p' argument.

   When you use `--diff', you do not need to use any of the other
options that relate to a local tree, specifically `--vc',
`--try-topfile', or `--try-topdir'. These options will be ignored. Of
course you must still specify how to get to the buildmaster (with
`--connect', `--tryhost', etc).


File: buildbot.info,  Node: Other Tools,  Next: .buildbot config directory,  Prev: Developer Tools,  Up: Command-line tool

8.3 Other Tools
===============

These tools are generally used by buildmaster administrators.

* Menu:

* sendchange::
* debugclient::


File: buildbot.info,  Node: sendchange,  Next: debugclient,  Prev: Other Tools,  Up: Other Tools

8.3.1 sendchange
----------------

This command is used to tell the buildmaster about source changes. It
is intended to be used from within a commit script, installed on the
VC server. It requires that you have a PBChangeSource (*note
PBChangeSource::) running in the buildmaster (by being set in
`c['change_source']').

     buildbot sendchange --master MASTERHOST:PORT --username USER FILENAMES..

   There are other (optional) arguments which can influence the
`Change' that gets submitted:

`--branch'
     This provides the (string) branch specifier. If omitted, it
     defaults to None, indicating the "default branch". All files
     included in this Change must be on the same branch.

`--category'
     This provides the (string) category specifier. If omitted, it
     defaults to None, indicating "no category". The category
     property is used by Schedulers to filter what changes they
     listen to.

`--revision_number'
     This provides a (numeric) revision number for the change, used
     for VC systems that use numeric transaction numbers (like
     Subversion).

`--revision'
     This provides a (string) revision specifier, for VC systems that
     use strings (Arch would use something like patch-42 etc).

`--revision_file'
     This provides a filename which will be opened and the contents
     used as the revision specifier. This is specifically for Darcs,
     which uses the output of `darcs changes --context' as a revision
     specifier.  This context file can be a couple of kilobytes long,
     spanning a couple lines per patch, and would be a hassle to pass
     as a command-line argument.

`--comments'
     This provides the change comments as a single argument. You may
     want to use `--logfile' instead.

`--logfile'
     This instructs the tool to read the change comments from the
     given file. If you use `-' as the filename, the tool will read
     the change comments from stdin.


File: buildbot.info,  Node: debugclient,  Prev: sendchange,  Up: Other Tools

8.3.2 debugclient
-----------------

     buildbot debugclient --master MASTERHOST:PORT --passwd DEBUGPW

   This launches a small Gtk+/Glade-based debug tool, connecting to
the buildmaster's "debug port". This debug port shares the same port
number as the slaveport (*note Setting the slaveport::), but the
`debugPort' is only enabled if you set a debug password in the
buildmaster's config file (*note Debug options::). The `--passwd'
option must match the `c['debugPassword']' value.

   `--master' can also be provided in `.debug/options' by the
`master' key. `--passwd' can be provided by the `debugPassword' key.

   The `Connect' button must be pressed before any of the other
buttons will be active. This establishes the connection to the
buildmaster. The other sections of the tool are as follows:

`Reload .cfg'
     Forces the buildmaster to reload its `master.cfg' file. This is
     equivalent to sending a SIGHUP to the buildmaster, but can be
     done remotely through the debug port. Note that it is a good
     idea to be watching the buildmaster's `twistd.log' as you reload
     the config file, as any errors which are detected in the config
     file will be announced there.

`Rebuild .py'
     (not yet implemented). The idea here is to use Twisted's
     "rebuild" facilities to replace the buildmaster's running code
     with a new version. Even if this worked, it would only be used
     by buildbot developers.

`poke IRC'
     This locates a `words.IRC' status target and causes it to emit a
     message on all the channels to which it is currently connected.
     This was used to debug a problem in which the buildmaster lost
     the connection to the IRC server and did not attempt to
     reconnect.

`Commit'
     This allows you to inject a Change, just as if a real one had
     been delivered by whatever VC hook you are using. You can set
     the name of the committed file and the name of the user who is
     doing the commit.  Optionally, you can also set a revision for
     the change. If the revision you provide looks like a number, it
     will be sent as an integer, otherwise it will be sent as a
     string.

`Force Build'
     This lets you force a Builder (selected by name) to start a
     build of the current source tree.

`Currently'
     (obsolete). This was used to manually set the status of the given
     Builder, but the status-assignment code was changed in an
     incompatible way and these buttons are no longer meaningful.



File: buildbot.info,  Node: .buildbot config directory,  Prev: Other Tools,  Up: Command-line tool

8.4 .buildbot config directory
==============================

Many of the `buildbot' tools must be told how to contact the
buildmaster that they interact with. This specification can be
provided as a command-line argument, but most of the time it will be
easier to set them in an "options" file. The `buildbot' command will
look for a special directory named `.buildbot', starting from the
current directory (where the command was run) and crawling upwards,
eventually looking in the user's home directory. It will look for a
file named `options' in this directory, and will evaluate it as a
python script, looking for certain names to be set.  You can just put
simple `name = 'value'' pairs in this file to set the options.

   For a description of the names used in this file, please see the
documentation for the individual `buildbot' sub-commands. The
following is a brief sample of what this file's contents could be.

     # for status-reading tools
     masterstatus = 'buildbot.example.org:12345'
     # for 'sendchange' or the debug port
     master = 'buildbot.example.org:18990'
     debugPassword = 'eiv7Po'

`masterstatus'
     Location of the `client.PBListener' status port, used by
     `statuslog' and `statusgui'.

`master'
     Location of the `debugPort' (for `debugclient'). Also the
     location of the `pb.PBChangeSource' (for `sendchange').  Usually
     shares the slaveport, but a future version may make it possible
     to have these listen on a separate port number.

`debugPassword'
     Must match the value of `c['debugPassword']', used to protect the
     debug port, for the `debugclient' command.

`username'
     Provides a default username for the `sendchange' command.


   The following options are used by the `buildbot try' command
(*note try::):

`try_connect'
     This specifies how the "try" command should deliver its request
     to the buildmaster. The currently accepted values are "ssh" and
     "pb".

`try_builders'
     Which builders should be used for the "try" build.

`try_vc'
     This specifies the version control system being used.

`try_branch'
     This indicates that the current tree is on a non-trunk branch.

`try_topdir'

`try_topfile'
     Use `try_topdir' to explicitly indicate the top of your working
     tree, or `try_topfile' to name a file that will only be found in
     that top-most directory.

`try_host'

`try_username'

`try_dir'
     When try_connect is "ssh", the command will pay attention to
     `try_host', `try_username', and `try_dir'.

`try_username'

`try_password'

`try_master'
     Instead, when `try_connect' is "pb", the command will pay
     attention to `try_username', `try_password', and `try_master'.

`try_wait'

`masterstatus'
     `try_wait' and `masterstatus' are used to ask the "try" command
     to wait for the requested build to complete.



File: buildbot.info,  Node: Resources,  Next: Developer's Appendix,  Prev: Command-line tool,  Up: Top

9 Resources
***********

The Buildbot's home page is at `http://buildbot.sourceforge.net/'

   For configuration questions and general discussion, please use the
`buildbot-devel' mailing list. The subscription instructions and
archives are available at
`http://lists.sourceforge.net/lists/listinfo/buildbot-devel'


File: buildbot.info,  Node: Developer's Appendix,  Next: Index of Useful Classes,  Prev: Resources,  Up: Top

Developer's Appendix
********************

This appendix contains random notes about the implementation of the
Buildbot, and is likely to only be of use to people intending to
extend the Buildbot's internals.

   The buildmaster consists of a tree of Service objects, which is
shaped as follows:

     BuildMaster
      ChangeMaster  (in .change_svc)
       [IChangeSource instances]
      [IScheduler instances]  (in .schedulers)
      BotMaster  (in .botmaster)
       [IBuildSlave instances]
      [IStatusTarget instances]  (in .statusTargets)

   The BotMaster has a collection of Builder objects as values of its
`.builders' dictionary.


File: buildbot.info,  Node: Index of Useful Classes,  Next: Index of master.cfg keys,  Prev: Developer's Appendix,  Up: Top

Index of Useful Classes
***********************

This is a list of all user-visible classes. There are the ones that
are useful in `master.cfg', the buildmaster's configuration file.
Classes that are not listed here are generally internal things that
admins are unlikely to have much use for.

Change Sources
==============