Posts Tagged perl

Quick IP prefix calculation

It’s a quite common task that you need to translate an IP address into a prefix — for example, when creating an IP prefix list from a set of addresses. Here’s a simple Perl script that helps it:

sudo apt-get install libnetaddr-ip-perl
cat >getprefix.pl <<'EOT'
use strict;
use warnings;
use NetAddr::IP;
if( scalar(@ARGV) == 0 ) {
    die("Usage: $0 PREFIX ...");
}
foreach my $pref (@ARGV) {
    my $ip = NetAddr::IP->new($pref) or
        die("Cannot create NetAddr::IP from $pref");
    print $ip->network()->cidr(), "\n";
}
EOT

# testing
cat >/tmp/x <<'EOT'
10.1.1.1/23
192.168.5.3/28
EOT

cat /tmp/x | xargs perl getprefix.pl | awk '{print "set ", $1}'
set  10.1.0.0/23
set  192.168.5.0/28


Advertisements

, ,

Leave a comment

Reusing HTTP connections in client-server applications

I’m working on a clientserver application which uses HTTP as a transport protocol for API requests, and sometimes there are occasions with a need to execute a few hundreds requests, such as data import or synchronization.

With default Apache HTTP server settings and default LWP::UserAgent options, every new request would result in a new HTTP session, and each time a DNS query is sent out. So, a synchronization process with a thousand object floods the DNS service with the same requests for the HTTP server name. This results in delays, and some public DNS servers apply rate limits which cause DNS lookup failures (had this with a domain hosted at Godaddy name servers).

HTTP 1.1 protocol supports reusing of persistent connections, but it’s not enabled by default in Apache and in the client.

In Apache HTTP server, the following options need to be configured:

  KeepAlive On
  MaxKeepAliveRequests 500

In the Perl client program, LWP::UserAgent needs the keep-alive option:

  my $ua = LWP::UserAgent->new(keep_alive => 1);

With these modifications, the DNS queries are only sent on every 500th API request, and the HTTP connection is reused between the requests, which saves CPU time on the server. This speeds up the whole process significantly, and also prevents the DNS failures caused by rate limiting.

,

Leave a comment

DENOG3 presentation

I made a presentation at the DENOG3 meeting last week and covered the following Perl-based open-source software products for network management and monitoring:

  1. Torrus, a well-established and mature software for massive SNMP polling and performance monitoring.
  2. Gerty, a new project for network automation. Any tasks on the network devices which need any interaction and automation, are targeted by the tool. The first release is expected soon.
  3. Mooxu, a new project which is currently in its early design phase. The product will provide a platform for distributed network testing and monitoring (eventually it may replace Torrus).

The slideshow PDF is available at the meeting agenda page, and also a video will be available soon.

, , , ,

Leave a comment

Distributed Testing Platform: design concept

Author: Stanislav Sinyagin
Document status: concept draft

UPD: the project name is now Mooxu

Introduction

In many network environments, especially in those of large ISPs or carriers, there’s a need to periodically execute some network tests. For example, an IPTV transport provider would need to make sure that all important multicast streams are available in every part of its edge network. Or a customer support engineer would need to collect byte and packet counters from a particular network port every 5 seconds.

The new software system (Project name: Mooxu) is designed to provide an open-source framework that enables the network operators to build the testing environment for their needs.  Also a number of open-source testing probe modules will be available.

Read the rest of this entry »

, , , , , ,

2 Comments

Autoconf/Automake installer for a Perl program

If you create a Perl application and want it to be installed in some standard part like /usr/local  or /opt/yourapp, it’s quite easy to do with a the standard GNU Autoconf, Automake and a couple of helper scripts. Read the rest of this entry »

, , , , ,

Leave a comment

using Tumblr API v2 from Perl

Tumblr API v2 uses oAuth mechanism for authentication, and this mechanism signs every request with a SHA1 or other hash algorithms.

If your posts contain non-ASCII data, here starts the tricky part. Perl has some special treatment for UTF-8 strings, and incorrect handling can easily lead to a wrong signature.

At first, I tried to use LWP::Authen::OAuth module, as it provides the simplest interface and hides most of oAuth internal logic. Unfortunately, it doesn’t care much about the stging encodings, and UTF8-formatted strings get corrupted and result in invalid signatures.

Net::OAuth appeared to be a bit more complex, and a bit insufficiently documented, but proved to work correctly with UTF-8 data.

Here’s the piece which finally worked. It took awhile to build this example, as the module documentation cuts some important corners.

Read the rest of this entry »

, ,

7 Comments