22.1. The NNTP Protocol

We've mentioned two NNTP commands that are key to how news articles are pushed or pulled between servers. Now we'll look at these in the context of an actual NNTP session to show you how simple the protocol is. For the purposes of our illustration, we'll use a simple telnet client to connect to an INN-based news server at the Virtual Brewery called news.vbrew.com. The server is running a minimal configuration to keep the examples short. We'll look at how to complete the configuration of this server in Chapter 23. In our testing we'll be very careful to generate articles in the junk newsgroup only, to avoid disturbing anyone else.

22.1.1. Connecting to the News Server

Connecting to the news server is a simple as opening a TCP connection to its NNTP port. When you are connected, you will be greeted with a welcome banner. One of the first commands you might try is help. The response you get generally depends upon whether the server believes you are a remote NNTP server or a newsreader, as there are different command sets required. You can change your operating mode using the mode command; we'll look at that in a moment:

$ telnet news.vbrew.com nntp
Trying 172.16.1.1...
Connected to localhost.
Escape character is '^]'.
200 news.vbrew.com InterNetNews server INN 1.7.2 08-Dec-1997 ready
help
100 Legal commands
        authinfo
                help
                ihave
                check
                takethis
                list
                mode
                xmode
                quit
                head
                stat
                xbatch
                xpath
                xreplic
For more information, contact "usenet" at this machine.
.

The responses to NNTP commands always end with a period (.) on a line by itself. The numbers you see in the output listing are response codes and are used by the server to indicate success or failure of a command. The response codes are described in RFC-977; we'll talk about the most important ones as we proceed.

22.1.2. Pushing a News Article onto a Server

We mentioned the IHAVE command when we talked about pushing news articles onto a news server. Let's now have a look at how the IHAVE command actually works:

ihave <[email protected]>
335
From: [email protected]
Subject: test message sent with ihave
Newsgroups: junk
Distribution: world
Path: gw.vk2ktj.ampr.org
Date: 26 April 1999
Message-ID: <[email protected]>
Body: 

This is a test message sent using the NNTP IHAVE command.

.
235

All NNTP commands are case insensitive, so you may enter them in either upper- or lowercase. The IHAVE command takes one mandatory argument, it being the Message ID of the article that is being pushed. Every news article is assigned a unique message ID when it is created. The IHAVE command provides a way of the NNTP server to say which articles it has when it wants to push articles to another server. The sending server will issue an IHAVE command for each article it wishes to push. If the command response code generated by the receiving NNTP server is in the “3xx” range, the sending NNTP server will transmit the complete article, including it's full header, terminating the article with a period on a line by itself. If the response code was in the “4xx” range, the receiving server has chosen not to accept this article, possibly because it already has it, or because of some problem, such as running out of disk space.

When the article has been transmitted, the receiving serve issues another response code indicating whether the article transmission was successful.

22.1.3. Changing to NNRP Reader Mode

Newsreaders use their own set of commands when talking to a news server. To activate these commands, the news server has to be operating in reader mode. Most news servers default to reader mode, unless the IP address of the connecting host is listed as a news-forwarding peer. In any case, NNTP provides a command to explicitly switch into reader mode:

mode reader
200 news.vbrew.com InterNetNews NNRP server INN 1.7.2 08-Dec-1997 ready/
    (posting ok).
help
100 Legal commands
  authinfo user Name|pass Password|generic <prog> <args>
  article [MessageID|Number]
  body [MessageID|Number]
  date
  group newsgroup
  head [MessageID|Number]
  help
  ihave
  last
  list [active|active.times|newsgroups|distributions|distrib.pats|/
      overview.fmt|subscriptions]
  listgroup newsgroup
  mode reader
  newgroups yymmdd hhmmss ["GMT"] [<distributions>]
  newnews newsgroups yymmddhhmmss ["GMT"] [<distributions>]
  next
  post
  slave
  stat [MessageID|Number]
  xgtitle [group_pattern]
  xhdr header [range|MessageID]
  xover [range]
  xpat header range|MessageID pat [morepat...]
  xpath MessageID
Report problems to <[email protected]>
.

NNTP reader mode has a lot of commands. Many of these are designed to make the life of a newsreader easier. We mentioned earlier that there are commands that instruct the server to send the head and the body of articles separately. There are also commands that list the available groups and articles, and others that allow posting, an alternate means of sending news articles to the server.

22.1.4. Listing Available Groups

The list command lists a number of different types of information; notably the groups supported by the server:

list newsgroups
215 Descriptions in form "group description".
control                 News server internal group
junk                    News server internal group
local.general           General local stuff
local.test              Local test group
.

22.1.5. Listing Active Groups

list active shows each supported group and provides information about them. The two numbers in each line of the output are the high-water mark and the low-water mark—that is, the highest numbered article and lowest numbered article in each group. The newsreader is able to form an idea of the number of articles in the group from these. We'll talk a little more about these numbers in a moment. The last field in the output displays flags that control whether posting is allowed to the group, whether the group is moderated, and whether articles posted are actually stored or just passed on. These flags are described in detail in Chapter 23. An example looks like this:

list active
215 Newsgroups in form "group high low flags".
control 0000000000 0000000001 y
junk 0000000003 0000000001 y
alt.test 0000000000 0000000001 y
.

22.1.6. Posting an Article

We mentioned there was a difference between pushing an article and posting an article. When you are pushing an article, there is an implicit assumption that the article already exists, that it has a message identifier that has been uniquely assigned to it by the server to which it was originally posted, and that it has a complete set of headers. When posting an article, you are creating the article for the first time and the only headers you supply are those that are meaningful to you, such as the Subject and the Newgroups to which you are posting the article. The news server you post the article on will add all the other headers for you and create a message ID that it will use when pushing the article onto other servers.

All of this means that posting an article is even easier than pushing one. An example posting looks like this:

post
340 Ok
From: [email protected]
Subject: test message number 1
Newsgroups: junk
Body: 

This is a test message, please feel free to ignore it.

.
240 Article posted

We've generated two more messages like this one to give our following examples some realism.

22.1.7. Listing New Articles

When a newsreader first connects to a new server and the user chooses a newsgroup to browse, the newsreader will want to retrieve a list of new articles, those posted or received since the last login by the user. The newnews command is used for this purpose. Three mandatory arguments must be supplied: the name of the group or groups to query, the start date, and the start time from which to list. The date and time are each specified as six-digit numbers, with the most significant information first; yymmdd and hhmmss, respectively:

newnews junk 990101 000000
230 New news follows
<[email protected]>
<[email protected]>
<[email protected]>
.

22.1.8. Selecting a Group on Which to Operate

When the user selects a newsgroup to browse, the newsreader may tell the news server that the group was selected. This simplifies the interaction between newsreader and news server; it removes the need to constantly send the name of the newsgroup with each command. The group command simply takes the name of the selected group as an argument. Many following commands use the group selected as the default, unless another newsgroup is specified explicitly:

group junk
211 3 1 3 junk

The group command returns a message indicating the number of active messages, the low-water mark, the high-water mark, and the name of the group, respectively. Note that while the number of active messages and the high-water mark are the same in our example, this is not often the case; in an active news server, some articles may have expired or been deleted, lowering the number of active messages but leaving the high-water mark untouched.

22.1.9. Listing Articles in a Group

To address newsgroup articles, the newsreader must know which article numbers represent active articles. The listgroup command offers a list of the active article numbers in the current group, or an explicit group if the group name is supplied:

listgroup junk
211 Article list follows
1
2
3
.

22.1.10. Retrieving an Article Header Only

The user must have some information about an article before she can know whether she wishes to read it. We mentioned earlier that some commands allow the article header and body to be transferred separately. The head command is used to request that the server transmit just the header of the specified article to the newsreader. If the user doesn't want to read this article, we haven't wasted time and network bandwidth transferring a potentially large article body unnecessarily.

Articles may be referenced using either their number (from the listgroup command) or their message identifier:

head 2
221 2 <[email protected]> head
Path: news.vbrew.com!not-for-mail
From: [email protected]
Newsgroups: junk
Subject: test message number 2
Date: 27 Apr 1999 21:51:50 GMT
Organization: The Virtual brewery
Lines: 2
Message-ID: <[email protected]>
NNTP-Posting-Host: localhost
X-Server-Date: 27 Apr 1999 21:51:50 GMT
Body: 
Xref: news.vbrew.com junk:2
.

22.1.11. Retrieving an Article Body Only

If, on the other hand, the user decides she does want to read the article, her newsreader needs a way of requesting that the message body be transmitted. The body command is used for this purpose. It operates in much the same way as the head command, except that only the message body is returned:

body 2
222 2 <[email protected]> body
This is another test message, please feel free to ignore it too.

.

22.1.12. Reading an Article from a Group

While it is normally most efficient to separately transfer the headers and bodies of selected articles, there are occasions when we are better off transferring the complete article. A good example of this is in applications through which we want to transfer all of the artices in a group without any sort of preselection, such as when we are using an NNTP cache program like leafnode.[1]

Naturally, NNTP provides a means of doing this, and not surprisingly, it operates almost identically to the head command as well. The article command also accepts an article number or message ID as an argument, but returns the whole article including its header:

article 1
220 1 <[email protected]> article
Path: news.vbrew.com!not-for-mail
From: [email protected]
Newsgroups: junk
Subject: test message number 1
Date: 26 Apr 1999 22:08:59 GMT
Organization: The Virtual brewery
Lines: 2
Message-ID: <[email protected]>
NNTP-Posting-Host: localhost
X-Server-Date: 26 Apr 1999 22:08:59 GMT
Body: 
Xref: news.vbrew.com junk:1

This is a test message, please feel free to ignore it.

.

If you attempt to retrieve an unknown article, the server will return a message with an appropriately coded response code and perhaps a readable text message:

article 4
423 Bad article number

We've described how the most important NNTP commands are used in this section. If you're interested in developing software that implements the NNTP protocol, you should refer to the relevant RFC documents; they provide a great deal of detail that we couldn't include here.

Let's now look at NNTP in action through the nntpd server.

Notes

[1]

leafnode is available by anonymous FTP from wpxx02.toxi.uni-wuerzburg.de in the /pub/ directory.