Downloading 1

---

Terminology 

    1. Article or Section - the number of characters posted as a unit - by Packet, the poster defined.  The number of lines can be altered, but most posters use either 7500 lines @ 76 characters each or 1/2 to 1 meg sized sections.  Articles can be reposted by themselves to complete a "missing section" request. See Missing Sections.
    2. Segment or File - the total single "message" posted.  Some posters only post 1 segment - " xxx.vid " and therefor have only one, others use " xxx.vid part 1 of 3; or xxxvid part 'a' ", where each 'part' of the three is a complete, viewable file.
    3. Archive (Rar, Zip or Split) - one of several means to divide up a file for posting. Freeware needed to recombine. See Viewing.
     4. Join - used in two ways:
                A. assembling the multiple parts / sections of a post as posted into usenet. Basic usenet tools needed (correct term is Extracting). See News readers.
                B. a means of viewing multiple files as one clip.  Freeware needed to recombine in some cases. See Editing.

Large binaries need to be split up into multiple, linked parts for posting.  The fractions indicate the part number.  For instance 5/32 is part 5 of a 32 article file.  You need all 32 articles available on your server to successfully download the section.  (See Missing Parts.)

If there's more than one n/n expression in the header, the first usually indicates the how many files are in a series, such as diddle_a.avi, diddle_b and so on.  Please note, some news readers might choke on these "false fractions". If you have problems, retrieve the message bodies individually, then select all the parts and use Manually Decode Binary Attachments to force your newsreader to join the parts correctly.

The 0/n (or 0/XX or 00) part is the beginning of the multipart post where the contributor includes a text message for you.  It usually contains system requirements, the needed codec or player, and a description of the contents.  A very high percentage of the questions posted in the ABM* newsgroups are the result of readers ignoring these introductory texts.

When you download you see "jumbled junk".  That IS the video, encoded into ASCII for posting to Usenet (see USENET).  If each line begins with an M, it's UUencoded.  If not, chances are it's MIME or base 64. see News readers and Decoding Off-line.

The following was posted by PolarBear, and I found it so good I clipped it as-is -

From: polarbear@bigwhite.paw (PolarBear)
Date: 13 May 2001 17:13:50 GMT

tolipa@ (ragnarok) wrote: I have asked about this problem in the past and cannot seem to get any help.  I download MPEG files and I get most of them, but others just appear as an endless jumble of data; sort of a txt file with no meaning. I read the FAQs but nothing points to this problem is, or how I might solve it.  Is it part of the news group community which accounts for all the requests for reposts?  I use Agent and have tried every way I could think of to recover these messages.  When I open them in Word Pad I see they are uuencoded with an "M" at the start of every line, but why does Agent not decode them to an MPEG but shows them as useless text? Is something missing? Was the download bad?

If you don't know what I'm talking about in the above, then please either ask, or go and read about it here.

When a poster posts an mpeg, his posting software typically does several things to the file, because most mpeg files are far too large to be posted as one piece to Usenet. Why? Because Usenet servers will only pass relatively small chunks of data between them, so files must be "chunked" in
such a way that they will propagate, and yet can still be re-combined by the viewer at the far end.

So, here's what happens:
The file, say it's a 20 MB mpeg binary, that is part "a" of a series, "Lola Lashes Los Angeles", posted by FooBar

1) is divided into "lines" (you don't need to worry about this).
2) the posting program makes up sections consisting of (in the case of Power Post 2000, for example) 7500 lines.
3) This process continues until the file has been completely parsed. There might be, say, 40 resulting sections, each one consisting of 7500 lines.

The file is then posted to Usenet.

Now you come along, somewhere else, on a different server. You see the file, "Lola Lashes Los Angeles". Being a big Lola fan, you gotta have it for your collection.

You see a header that says something like:
256418 <NEW> Lola Lashes LA - 1 of 5 - FOO_LolaLA-a.mpeg (*/40)
And you download it, and it uudecodes and plays. This is because the file is complete - all the posted parts are there, and (we hope) uncorrupted. The next day, you see another post, by FooBar, for part "b"
|__| 90411 <NEW> Lola Lashes LA - 2 of 5 - FOO_LolaLA-b.mpeg (*/20)

Now, there are two things about this that are striking. One is that there is a funny shaped icon to the left, which indicates an incomplete binary file. The second is that the file length is _small_, compared to the earlier file. Same poster, same software, and most posters tend to cut their videos fairly consistently.

So if you download this, you will get garbage when you try to uudecode it. If you click on the message header, you will get the following message:
90411 lines, incomplete
This is a multi-part message, of which only 7 of 20 parts are available. These parts are unavailable: 1, 2, 7, 8, 9, 11, 12, 14, 15, 17, 18, 19, 20
To retrieve the incomplete message, use the menu option Message|Join  Sections.
Now, this menu option is a bit misleading. What it does is _save_ the parts that are available NOW, and keep them until you can find the missing ones. There's no magic about it that turns the incomplete message into a whole one.

When you are able to download the missing parts, Agent can then store them in the same folder, then combine them and uudecode them to produce a complete file. Not before.

Hope this helps a bit.
PolarBear 

Decoders and Configuration:

There are five "formats" for encoding files that may be posted to usenet - UUEncode, Binhex, XXE, base64 (MIME). and (as a part of mime) - yEnc.

While many news readers can handle most, if not all, of these, you may find yourself with a file that won't decode in yours.  The below is a list of Decoders that you may need to add to your system in order to decode them. Configure your decoder to join multiparts (concatenate).  Occasionally, a Usenet poster will bundle several files into one MIME post (not in ABME, please!!!) and a good decoder will extract all of them if configured correctly.

Here are some MANUAL shareware decoders to try out:

o XferPro Shareware: from Safasoft (Windows and Macintosh) Just double click on any file you want to decode.  XferPro will create the new file with the correct filename in the same directory and then close by itself.  If the file size says 0, not to worry.  That will right itself after refreshing the view or using the file.  Try to play the video or extract the zip file you just decoded.  If all goes well, delete the encoded file(s).

o Wincode (Windows only):  Many like the standard Windows GUI (graphical user interface) and most Usenetters are familiar with it, if that's any comfort.

o CoderPad (Windows only):  Standard features, plus cut and paste input directly from your newsreader.

o Decode Shell Extension - This Windows Extension utility adds the ability to the Windows Contextual window ( most often referred to as the window that appears when you right click on a window, link, or icon) that will allow you to manually decode a UUEncoded, Binhex, XXE, or Base64(MIME) file that you have saved manually.  Great for dealing with those pesky Base64 files that won't seem to open or decode.  Plus, it's free.

o MindExpander - free from MindVision, the very same company that created the WISE Installer utility used by programmers that you have undoubtedly been exposed to as you've installed new software. MindExpander has become popular due to its ability to effortlessly handle .SIT, .SEA, .HQX, .BIN, and .ZIP.  It's definitely worth a download, for both Windows and Macintosh

A zip archive can contain anything, including executables.  You can't go strictly by what the Usenet subject header says.  Get a zip extraction utility with a graphical user interface, such as WinZip (archive sites), and learn what all the commands do.  For instance, don't assume you know what "check out" means.  You can safely decompress the contents of the zip archive, but don't run any of the executable files until you've performed a virus scan on them.  If a virus is present, the only way you could activate it is by executing the file.

- the new posting format. yEnc  is an 8-bit text format built with the mime style - thats how it achives its space savings on servers. Most of the older newsreaders do not understand 8-bit text, they only understand 7-bit text. When we "encode" items in binhex, yEnc, Uue, base64, etc, we are taking one form of file (a binary in this case) and turning it into raw text, (the encoding) for posting onto USENET. Older methods of encoding are 7-bit, and they generally have around 30% or more "overhead" in the text conversion than yEnc. That adds to the size of the post in uploading, news server storage, and downloading. yEnc only adds 1-2% of this "overhead", making it an ideal USENET  transmission method. Most usenet feeds charge by the amount of volume you upload / download, switching to yEnc encoding would reduce the amount of time / bandwidth required for this. While many of the current Newsreaders (Xnews, Gravity, MyNews) either can decode or have add-ons to allow automatic de/encoding, the top two Newsreaders in use - Forte FreeAgent and Outlook Express Do Not.

Posting conventions for this encoding allow for movie.001, movie001.ntx, or 001-movie.ntx. For FreeAgent and Outlook Express users hopefully all that ever will be posted are the "ntx" marked ones, so that they'll be easy to spot, but if you encounter a file that won't decode in freeAgent or Outlook, save the file as a text file (strange-one.001.txt) and look at it with a text reader like notepad. If you see:
=ybegin part=3 line=128 size=25431 name=movie.jpg
=ypart begin=10241 end=15360
in the first two lines, you'll know you've downloaded a yEnc encoded post.

Note to OutExpress users. IF you flat won't switch, you'll need a tool like yEnc Proxy to allow you to "read" the yEnc posts. Follow the instructions posted on their page. An outline of the process can be viewed here

an OS 9 solution for yEnc, WinRAR.3 and Par files
From: Alephander <alephander@not
Virtual PC works, even though VPC4 has trouble quitting under OS9. But, you'll need a newsreader which won't try to translate char sets on receiving messages from the server, which all the older Newsreaders attempt to do.

for OS X.
winston <noone@nowhere.invalid> wrote:
 If you use MacSOUP's save as raw text feature you can download the segements and then decode with Rosetta or yEncTZ. The  reason yEncTZ did not work most likely, is because MacSOUP was not   downloading the segment's 8-bit text properly. To do so with MacSOUP, the only thing you have to do is select the articles got to File -> Save Articles and select the option rnews batch. After doing that you can encode it with yEncTZ.

The Macbinary wrapper is a feature Thoth uses to wrap the yEnc binary segment into a shell, which protects the meta-data and creator codes and such, because normal yEnc transmissions lose that data. (which is why binhex is so popular, even tho it has such overhead - it keeps the creator codes, etc, also, and almost every mac newsreader available supports binhex downloading and decoding because it is a 7-bit,
many-years-old standard encoding.)

A list of  Mac Newsreaders which -can- receive 8-bit encoded (eYnc) messages as of 08/02/2002:
Thoth 1.5.1 and later
MT-Newswatcher 3.2 (avail OS X only), using yEncTZ or Rosetta for decoding
Halime 0.8.1 and later (avail OS X only)
Halime, (avail OS X only)
Hogwasher 3.0 (released Jul 1 02)
MacSOUP 2.4.6 : yes (save as unix mailbox) 
YA-Newswatcher *all versions* : no

For yEnc decoders / encoders for all systems, check the links here

Down loader's Questions, Answers, Comments, Requests 

Small files at the begining of posts are often "0" files. They can end in *.txt or more commonly to prevent problems with servers that will not carry txt files in binary groups - .nfo. These texts contain information about system requirements, codecs, passwords, and number of posting days, so please get into the habit of checking them.

An overly high percentage of the requests for reposts, and reposts of reposts, are from people who don't check ABME often enough to catch the files they want.  Please be fair to the posters and other readers.  When you ask for a file, check back daily to be sure you don't miss it.  If you can't or won't keep up with the group, then don't post requests.

Can't get these files to work?  Have you read the "0" file? When you ask for help, provide enough information so someone CAN help you.  At a minimum give your OS type (Win9x, Win 3.1, Mac, etc.), your CPU (pentium, 486, etc.), a description of your problem (uploading, downloading, decoding, viewing), and the specific file you are having trouble with.  The more information you give, the better the chances someone can provide you with an answer.

The d groups are where you'd find timely announcements about server problems, corrupt posts, new codecs, virus alerts, etc.  Scan the headers before posting a question, it may have been asked and answered already.  If you have Agent, finding your answers is actually quicker than asking.  Select the pertinent newsgroups and then use Edit | Global Search to find posts discussing whatever you need to know.  This feature is not built into FREE Agent, but is available as a separate utility called "Agent Global Search".

Why some people who ask questions get ignored.

1. Polite questions in the proper groups get responses, unless the answer is already staring them right in the face.
2. The poster might be ignored or get a little reminder to read before before asking.
3. A question goes unanswered because not enough information is provided.
4. The question should have been directed elsewhere, such as to a software or hardware vendor, or internet service provider.
5. Chat and requests in ABME itself are mostly ignored because replying would only reinforce and perpetuate the non binary posts to that group.
6. Most of the contributors filter out the chat in the e group.
7. People who are rude and abusive or who post garbage are called out on it.

Is there any place else to go? Well, how much do you want to pay? Nothing? then no, not really. Every time anyone attempts to duplicate the content of abme/abmne they find they must charge, or the service falls apart. But there is a small hope - if you're a true collector and have something to offer in exchange. There is an napster like exchange system called "the Hub". You'll need to already have a certain volume of files to offer. The starting point for learning about the hub is here: http://www.abme.to/hub/index.php

ISP's & Supplemental News Servers

  A short list of ISPs can be found here.  If your server misses lots of posts, consider getting a different ISP or subscribing to a supplemental news feed (NSP).  Excellent news feed means better than 85% complete posts within 24 hours of posting, including multipart in chunks of 10,000 lines or more, and getting all parts of huge files.

Good retention is at least 3 days, often longer.  You can see which ISPs have Dial Up access in your area at the list or the directory.  Be aware:  Many ISP's use "news farms", and each time you log on you get a randomly assigned server. Occasionally, that machine might not be correctly synchronized with the others on the news farm, so it does not give you messages you should be getting. Suspect you have this problem if some posts "expire" prematurely or if you are missing parts that you know other members have retrieved.  Ask your provider how to force a connection to the same machine every time.

 A long list of supplemental news feeds, both Free and Fee based are here.  Don't be fooled by services with high numbers of newsgroups carried, since thousands of groups are defunct. Also watch for "completes", as many news feed services may carry thousands of groups, but have almost no complete binaries in any group.

Your speed will vary with each based on its' remote (vs. local) server connection, so make sure it will be usable for you BEFORE you sign on.

 Do a ping to the server and look for packet loss.  Do a trace route as well.  A few of those, at different times, and you'll have an idea of what your upload times will be. If your main thing is downloading it matters NOT how fast the trace route times from you to them are - nor how few the hops TO their servers. It's the FROM that counts. Without a utility on their end to demo that speed, you're buying a pig in a poke... and some of these services are pigs, indeed.

Check to see if the News Feed has a utility on their end to demonstrate the speed of d/l to your location. Since trace routes are different to and from, without this you'll have absolutely NO idea how fast the service will be - until after you've already paid the $$$. Blake found a great utility for testing nntp newsfeeds called "NetStat Live"? It's a software to monitor more than just your download and upload speed. I down loaded a 10meg clip and used NetStatLive to monitor the download and whoa! up and down the graph went. It would peak and dip like an SOB, I say the service at that NSP was very inconsistent. I then downladed the same clip via another NSP and monitored the download using NetStatLive and the difference is clear. Speed went up and stayed up.
NetStatLive by "AnalogX"

Web (http:) based servers.A growing trend, coming from the number of people unable to comprehend the differences between the different protocals of the internet. Some of the nntp hosts like Newsguy and GigaNews also have complete access to their files through an http connection. Others are attempting to offer http newsfeeds as part of their service. Look for this growth of these CSP's (Content Service Provider) to continue, and might in fact be trend of the future.

Connectivity Problems 

 Whenever you try to download a large file, either your modem quits or your server disconnects you before you're finished and you have to start all over.  You try to download a bunch of videos while you're sleeping or at work, only to come back and see is a screenful of error messages.  There's a better way.

Hunt around for connectivity, TCP/IP and newsreader tools at the Archive Sites.  What you're looking for are utilities to redial a disconnected modem and to restart your newsreader tasks.  One word of caution, though:  some "connectivity" tools are just keep alive pingers which might violate your ISP's Terms of Service.  That's not what we're talking about anyway.

  1. For modem disconnection, get DUNCE (Dial Up Networking Connection Enhancement).  It works: by automatically entering "connect" whenever your DUN dialogue appears.

  2. For Time outs using Agent (not free agent) - Task Monitor .  In the event of a time-out or disconnection, restarts retrieving marked message bodies and posting Usenet and email messages.  Use in conjunction with DUNCE for foolproof downloading.  How it works:  Searches at specified time intervals for error windows where the parent window contains the word "Agent."  If you have marked messages for retrieval in multiple newsgroups, Task Monitor will resume downloading in news group order.

   3. To rearrange Messages to your liking, get Agent Group Order, or, in Agent:
    a) click on Options | User and System Profile | System,
    b) check the box titled "Server Creates messages out of order."
In Free Agent,
    c) click on Group | Default Properties | Retrieving,
    d) check the box titled "Server Creates messages out of order."

   4. To keep from timing out so fast, try changing your agent.ini settings. Extra foolproof downloading:

[Timing]
GetHostByNameTimeout=1000
ConnectTimeout=1000
ResponseTimeout=1000
PostResponseTimeout=1000
QuitResponseTimeout=1000
SendTimeout=2000
PostTimeout=2000
SplashTime=2
SMTPSessionTimeout=1000

a). Select the file parts and download the bodies first (get marked message bodies).
b). After you have all parts of the file, click on part 1 and save it. The advantages of this method are that you can stop and resume your downloads at any time.
c). In case of a disconnection, you only have to resume from the last part retrieved.  The only disadvantage is that you will need sufficient hard disk space to temporarily store many large encoded files during a long download session.

(1) your modem connect speed (not to be confused with the max speed your modem will handle)
(2) what news server you are downloading from (i.e., LOCAL, your ISP's news server, or REMOTE, another news server you connect to through your ISP, such as Newsguy or Airnews).

Notes on Road Runner News Servers 

 

---

AOL - a class by itself?

Oh, well, some notes about Filegrabber:

IN AOL land, if Filegrabber won't give you the "download file" option it usually means you're missing parts of the post on your server.  If all the parts ARE there, then the post is mime or the Filegrabber just isn't working (see Decoding Off-line). To find only complete multipart posts, AOL has given you the option of filtering out all text posts and incompletes.  The downside of this feature is that you won't be able to read part 0/n of the posts, which contain important information.  If Filegrabber say the message is too long for downloading, keep clicking the More button until the whole body of the post is in the window.

If your Filegrabber doesn't give you the option to download the file (as opposed to the article), the post is either MIME or an incomplete UUE post.  You must have all the parts and download them individually.  After downloading, examine one of the files in any text reader to determine whether it's MIME or UUE.  Rename the files accordingly, then decode.

     (Free)Agent or other news readers on AOL cannot be used to access AOL's news server, but you can use them with a supplemental news service from your AOL connection.  See Help | Release Notes | Secure News and Email Log ons.

AOL's built-in decompression utility can't handle passwords, and it's unwise to use it anyway.  Click on Members | Preferences | Download and remove the check marks from  "Automatically decompress files at sign off" and "Delete zip files after decompression."  Get WinZip or another good decompression utility and learn how to do it yourself.

There are "features" of the AOL newsreader that will often give you an error message stating that "Follow-up is allowed via email only."  This is dead wrong, and not the author's intention at all. Your news reader is misinterpreting the "reply to" and "follow up" fields in the original post.  In other cases, you are able to post a follow-up Usenet message, but only to the same newsgroups as the original post and you can't redirect it.  Please hold off posting any messages until you have closed ABME and switched to ABMED. There is a very complete, if old, faq for usenet on AOL by Gordon202 you might want to read. The Large Binary Section link is with the AOL link.

---

This Web Site and its contents - ©2000-2004 abmefaq consortium