systhread.net : Previous Changelog

Changelog

Fri Aug 13 19:31:59 EDT 2010

Site News for 3rd Quarter 2010

The general frequency of the site has been 1-2 times/month. Recently this was changed to a full quarter. The full quarter idea does not seem to work well. Too much change. That said the frequency is likely to change to an average of once/month. Meaning a month could pass with no news and then a month with two items and so forth. Otherwise absolutely nothing here has changed... which is of course a good thing. Quite a few items on the plate so read on.

Enlightenment Transform Utilty (`etu`) `0.1.8` Cut

A lot of changes with this release of the one and only graphics program I maintain. No remarkable user changes though so if your installation still uses the epeg library then there is no need to upgrade. That said, if you are tracking enlightenment then the current version will not deal with jpeg image formats at all and may be using legacy libraries (if it actually works). Following are the changes made to this version:

Migrated source over to git again (my first pass at this last year did not work right).
Moved epeg functions to imlib2 (where they now reside in e17).
Ran through the valgrind harness.
Added file information option.

etu · Coding

Replacing Ping with Nmap for Nagios

From the text:

Sometimes a system administrator needs to get around a few rules that are in place for good (or not) reasons. One example is when networks have ICMP turned off (or even just a portion of it). With ICMP off it can be difficult to configure tools like Nagios for simple up and down checks. In this text getting around the no ICMP problem and a script to handle it for Nagios.

Text

In 2600 2010 Summer Edition

Another article written by yours truly is in the 2600 Magazine. The article is a 10,000 ft. overiew about how to setup personal darknets. Eventually material written for 2600 may make its way here. Some in fact already has. This is due to 2600's excellent republish policy which states once 2600 prints it the rights revert to the author.

2600 · 2600 2010 Summer

New Feeds for the Site

It only took five years but finally, for those interested, there are external feeds/pages about the site for those who do not directly suck down the RSS file. They are:

As per the norm if it turns out the feeds/other sites are more or less useless they will be tossed or alternatively simply forgotten.

Taking a crack at Passive Scanning

Probably the most interesting and incomplete project at the moment is Netreconn now has the beginning(s) of a passive scanner. So far the lesson learned has been while snarfing ports and enumerating them per host is easy enough, there are a lot of challenges when using pure passive taps to scan for hosts... which is not really what is going on. What really is going on is the wire is being watched and particular data is being correlated. Regardless here are a few of the challenges thusfar (anyone interested may feel free to download the code and have a look):

Ports need their own data structure to record protocol name(s) and port number. This is a pure laziness issue and I will get to it.
The first pass at OS determination will be via port combinations. I have no idea how that will work.
Full fingerprinting has been requested. Not sure how to do that yet or even if I want to.
structures need to be sorted. I am saving this for last because I don't know what the structures are yet.
How to determine a real service vs. a client port? My thinking right now is N hits and different clients accessing a common singular port. Again, I've no idea how I will implement this.

Otherwise it works, that is to say it can be a bit wily nily but the core engine that gets the data is there. Eventually the plan is to merge all three utilities into one. So scanlan, wiretraf and passive would be one shared codebase. The exception is I intend to leave a make target to build scanlan (via defines) with no depends so users can just copy the static binary anywhere they need to if they do not have pcap libraries available.

Sun Jun 13 12:30:46 EDT 2010

Site Cleanup & Updates Done

The aforementioned site work is complete. Not really all that thrilling. Following are some of the chores wrapped up:

Compressed the news page.
Put several new series in the series lists on the texts index.
Updated the about section.
Updated the coding section (nmap stuff)
Changed the site content license to CC. There is a good reason for that ...

New Book Available

Last year I put together a book with selected texts from the site and some new material. The topic is basically the same as most of the site content regarding programming. After peddling the draft around I finally decided I didn't have the energy to keep packaging it up along with supporting materials anymore. Instead I decided to just give it away under a Creative Commons license. If there is any interest in the book (and that is a big if) I might do another one packaging up all of the site material as a sort of reference/history. The working title is simply System Utility Programming and can be perused in a variety of formats:

All of it Downloads

PDF, HTML, PS, PDF and Sections in HTML all in one tarball.
Target Formats plus Make bits, helper scripts and source text files

Book Formats

Additionally I broke it up into the major sections:

Part 0 · Part 1 · Part 2 · Part A

The cover can be found here for those who might want to print out the entire book.

texts · sysutil-book

New Version of `netreconn` Available

I have branched a new version of the netreconn tools. There have been some major changes to it and there is still a lot of work yet to go. Following is a list of major changes:

collapsed TODOs into the top of source files
collapsed the pcap programs (ndecode, arpsniff and ntraf) into wiretraf
moved nstrobe to scanlan
Removed ntrace script
Removed nlist script
Print start and stop time at end of scan
Added arp traffic reading

Here are a few of the TODOs. As per the norm some, none, all or totally different things may happen to the utilities:

`scanlan` TODOs

add session trace (only one level, none of this d1-N stuff)
add udp support
different socket type support (e.g. raw), look at how dnet does this
perhost timers with -vv option or *something else*
if practical a true pre-ping using ICMP versus a full connect
Support for user to change TCP flags in both directions

`wiretraf` TODOs

some explanation of the output fields
arp needs to have timestamps
arp needs decoding
ethtraf (will want src/dst mac + IP)?
traffic counters both total and as packets come in (ARP has the latter)

The git repo has all updated sources now for netreconn as well.

netreconn-1.77 · coding

Nmap & `Dnet` Work

In addition to all of the other stuff I have been up to lately I managed to find time to wrap up a small Nmap project and complete (at least as far as my infrastructure supports it) a big chunk of Dnet work.

Nmap: After several months of a few hours a week hacking (prefaced with a couple of months of dialogue) we finally moved payload definitions out of the source code and into their own file. The file is parsed at runtime and loaded into a std::map for payload lookups. What does this mean? If one wishes to use a new payload all they have to do is add it to nmap-payloads versus adding it to the code and recompiling. Currently only UDP is supported.
Dnet: In libdnet land I wrapped up all of the changes from the stripped version that nmap uses. This actually fixed a handful of bugs. Most notably was a bug where an interface name could be missed by one of the dnet routines. I am not sure how close we are to a new version but I would like to crank one out this year if possible. We shall see.

Sun May 16 18:30:00 EDT 2010: Site News

Of course expect this entry to be deleted soon. Over the next month I might be taking a break from writing to perform some content maintenance. This is what happens when one does not use a database. They have to clean stuff. Specifically the news needs compressed into simple lists (which has to be done manually... great). Also the texts index page needs some new series lists put together and in of itself might need to be split (I haven't really decided yet). No fear, I do have some interesting content on the horizon (in the form of notes) I just need to sit down and actually, you know, write it. I think the about section could use a punch in the arm as well but we shall see. I do not plan on changing the design, just content whereabouts, lists and so forth. Honestly the design took me so long to settle on and is so complex at this point, I am kind of scared of even looking at it.

Fri May 7 20:45:00 EDT 2010: Nagios Meta Check Part 3

In part one of this series the basic trusses needed by the Nagios check_systemhealth script were put together. In part two the actual checks themselves were coded. In this the third and final part of the series compulsory checks are added, the main loop is constructed and the final full source listing produced.

It is worth noting that this is only one of many methods to achieve the same goal. There exists at Nagios exchange plugins and scripts that can do similar actions such as aggregate groups of checks, services and so on. The code presented in this series is just a touch upon a single idea designed to make the reader think about their monitoring deployment.

Text

Fri Apr 2 11:36:20 EDT 2010: Going (somewhat) Retro on Unix

Yes I posted this today to avoid the April Fool's joke wonderings. Recently during a short period of severe boredom I decided to try and change my habits a bit by using - when possible - nothing but command line tools. I did allow for the use of curses based tools too, so I guess console or terminal only tools would be a more appropiate way to state the experiment. Many of the tools I already did use but I wanted to see if I could use exclusively console commands/tools/utils for a week or so. The result was pretty surprising, excepting Firefox (which I found a retro skin theme for) and audacious (for streaming internet music stations) I still use nothing but console utilities in my Xsession and am still using the window manager I setup. Note this is not a review of tools or anything like that, just an experiment that had some unexpected benefits. I am thinking about trying the opposite but I fear it won't be nearly as interesting.

Text

Wed Mar 3 17:42:26 EST 2010: Using Nmap to Fix a Problem

Ever had an ipv4 network address that is supposed to migrate over via a high availability mechanism simply not work or even stranger if there were several addresses some do and some do not? An experienced network administrator probably has seen mysterious non-migrating addresses, however, within this context is presented a rather interesting "solution" to when it has been observed.

Text

Mon Feb 8 21:00:46 EST 2010: `netreconn-1.76` & `mmw-2.0`

netreconn

Finally got around to releasing the stable version of the netreconn utilities. These are basically the same as 1.75 without any changes. I think I am going to stop using the odd numbered/even numbered method since there do not seem to be enough changes in between to bother.

netreconn

mmw

I was really bored one day and finally did some work on the micro memory watcher or mmw. The mmw utility is basically a nicely formatted version of free. Following are the changes in this version:

Updated manual page (finally)
Added a subdivision of 1/10 GB (not apparent to user) so ranges of 1-10 GB still print in MB units
Changed usage over to an atomic printf
Converted exits and returns to posix macro
Fixed if no /proc/meminfo to exit with failure (before it did nothing)
Moved the sleep interval to end of reading /proc/meminfo to mimic how other similar utils work
Combined poll and sleep check into one shot deal

mmw