The following bash script iterates over dates in a range.

#!/bin/bash   # $1 = start date (e.g.: yyyy-mm-dd) #$2 = end date   # make sure the end date is formatted correctly end_date=$(date -d "$2" +%Y-%m-%d)   # set iteration date to start date and format iter_date=$(date -d "$1" +%Y-%m-%d)   until [[ ${iter_date} ==${end_date} ]]; do # print the date echo ${iter_date} # advance the date iter_date=$(date -d "${iter_date} +1 day" +%Y-%m-%d) done I’ve saved that in a file called “loopdates.sh” and chmod the file to 0755. An example usage follows. $ ./loopdates.sh 2014-11-27 2014-12-02 2014-11-27 2014-11-28 2014-11-29 2014-11-30 2014-12-01

Using the ‘du’ and ‘sort’ commands, we can get a listing of the largest directories in a given directory. For instance, in my home directory:

jason@mintSandbox ~ $du -h --max-depth 1 | sort -hr 7.3G . 4.7G ./data 1.4G ./mount0 391M ./.cache 389M ./bash 258M ./tmp 104M ./texts 35M ./python 28M ./.mozilla 12M ./.adobe 11M ./.config 8.8M ./bin 8.3M ./Downloads 3.7M ./.thumbnails 1.8M ./presentations 1.3M ./.macromedia 884K ./.gstreamer-0.10 524K ./.gimp-2.8 508K ./misc 388K ./.purple 164K ./.local 152K ./.java 144K ./.netExtenderCerts 140K ./.kde 72K ./scripts 64K ./.gconf 36K ./.pki 36K ./.gftp 32K ./.gnome2 28K ./.ssh 16K ./.linuxmint 12K ./.dbus 4.0K ./mount2 4.0K ./mount1 4.0K ./.gnome2_private 4.0K ./Desktop Here, the ‘du’ command (standing for ‘disk usage’) estimates disk space taken by directories. The ‘-h’ option tells ‘du’ to make the output human readable. The ‘–max-depth 1′ option tells ‘du’ not to dig down within folders. (Try issuing the command without this option and see what happens.) The ‘sort’ command then takes the output of ‘du’ and sorts it by human readable numbers (again, the ‘-h’ option). The extra ‘-r’ option simply tells sort to reverse the sort, so the largest folders come first. We could pipe this to ‘head’ to reduce the number of rows returned to, say, 5 rows: jason@mintSandbox ~$ du -h --max-depth 1 | sort -hr | head -n 5 7.3G . 4.7G ./data 1.4G ./mount0 391M ./.cache 389M ./bash

Say you have a bunch of subdirectories of your current working directory, all including variously named files. You want to iterate over those files and apply some Bash command.

For instance, I have folders named 01, 02, 03, …, 31 (representing days in a month), and inside each of those folders sits various files. I wish to gzip each of those files individually. Here’s how I do that with a single line in Bash:

$for d in */; do for f in$d*; do echo "gzip ${f}";gzip${f};done; done

The [date] program in Linux is incredibly powerful, and can be used to modify dates very quickly. Here are some examples.

The current time in my present locale:

$date Thu Nov 13 15:41:12 MST 2014 The current time in UTC: (Use UTC, not GMT. One is an international standard of keeping time based on atomic clocks while the other is a local old-fashioned timezone based on when the sun is highest in the sky … which isn’t exactly accurate enough for international business. Plus, GMT isn’t even used when daylight saving time is in effect. Anyway…) $ date --utc Thu Nov 13 22:44:06 UTC 2014

The time one day ago in UTC:

$date --utc -d "now -1 day" Wed Nov 12 22:44:51 UTC 2014 A specific date minus one day, formatted as we wish: date -d "2014-10-01 -1 day" +%Y-%m-%d 2014-09-30 See the manual page for date for more options. Say you want to transfer a large file over your network with scp, but you don’t want to hoard all of the network resources for this transfer. You can limit scp’s bandwidth usage by using the -l (lower case L) option and specifying your bandwidth limit in Kbit/s. So, if I want to transfer a file and limit the bandwidth used to 1MB/s, I’d first compute that 1MB is equal to 1024KB, which is again equal to 8192Kb. (Here B=Byte and b=bit.) So, we end up with the command: $ scp -l 8192 file_to_transfer user@host:/path-on-other-end

When a query is running in MySQL, there are several options for killing it. One is to it, which isn’t recommended. A better way is to use MySQL’s built-in ‘kill’ command. Log in to another instance of MySQL, find the process number of the query, and kill that process. It looks a bit like the following after logging in.

mysql> show processlist; +--------+--------+-----------+-----------+---------+------+--------------+--------------------------------+ | Id | User | Host | db | Command | Time | State | Info | +--------+--------+-----------+-----------+---------+------+--------------+--------------------------------+ | 222954 | rouser | localhost | DATABASE | Query | 3945 | Sending data | SELECT COUNT(*) FROM BIG_TABLE | | 223347 | rouser | localhost | NULL | Query | 0 | NULL | show processlist | +--------+--------+-----------+-----------+---------+------+--------------+--------------------------------+ 2 rows in set (0.00 sec)   mysql> kill 222954; Query OK, 0 rows affected (0.00 sec)   mysql>

Many times in a Unix environment, the file extension of an executable is removed and Vim/Vi may not pick up on the language used. Forcing syntax highlighting in a specific language is easy. For instance, to set the highlighting language to C:

:setf c

For bash:

:setf sh
Posted in Vim

## Problem:

You can connect to MySQL on computer #1 and you can connect from computer #2 to computer #1 via SSH, but you cannot connect directly from computer #2 to computer #1 with MySQL – and you want/need to do so.

## Solution:

Create an SSH tunnel to route the MySQL connection. Here’s how.

On computer #2: Run the following command in a terminal:

ssh username@computer1address -L 3307:localhost:3306 -N

1) “-L 3307:localhost:3306″ means that we want to bind port 3307 on the local machine (computer #2) to port 3306 on the remote machine (computer #1). After the connection is made, connections on computer #2 going to localhost:3307 will actually be connecting to computer #1′s port 3306.
2) “-N” means to not expect anything to happen after the connection is made. It will just sit there and allow traffic in the background.

Now, you should be able to connect from computer #2 to MySQL on computer #1. On computer #2, now try:

mysql -h 127.0.0.1 -P 3307 -u username -ppassword

In the context of a program, an Easter egg is an undocumented response that results from a specific input. The classic example is the Konami code (up up down down left right left right B A), used as a cheat code in various video games. (The Konami code has been used in many other places you wouldn’t expect, including a lens flare effect that used to be an Easter egg on Facebook.) You can Google that, so do that if you haven’t already. Here, I want to list a few fun Easter eggs.

I started this by simply looking at Python, but then expanded the list to include some others. So, it’s still a bit heavier on Python with randomness coming afterwards.

## Python

### import braces

Python has some good ones. My favorite is the one that is built in to the ‘__future__’ module. This module is intended for future Python additions, or things that are going to be in Python but aren’t there just yet. One thing that keeps getting asked for by some Python users is a C-style braces-wrapped scoping system, contrary to Python’s white-space scope delimiters. So, when ‘braces’ showed up in the ‘__future__’ module, some may have been excited … until they tried it out. Here’s what happens.

>>> from __future__ import braces File "<stdin>", line 1 SyntaxError: not a chance

### import antigravity

This references an XKCD comic. It actually opens a browser and loads the XKCD comic when you run it.

>>> import antigravity

### Hello World

There’s a hello world Easter egg in Python.

>>> import __hello__ Hello world!

### The Zen of Python

This is probably, at this point, the most well-known Python Easter egg. “The Zen of Python,” now officially PEP-20, was originally sent by Tim Peters (now functioning as “occasional contributor of great knowledge” to Python) to the python-list back in 1999 in this post. In 2001 while organizing IPC10 (the precursor to PyCon), Barry Warsaw and Tim filtered through a massive list of crowd-sourced potential slogans for an IPC t-shirt and came up with “import this”. The combination of “The Zen of Python” and “import this” was merged secretly in Python 2.2.1, and if you ‘cat’ the contents of this.py (e.g., in /usr/lib/python2.x) you’ll see that there is some encryption so the message isn’t immediately obvious in a commit. Anyway, it does the following when called.

>>> import this The Zen of Python, by Tim Peters   Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those!

## Firefox / Iceweasel

If you’re using Firefox (or its Debian brother with properly licensed logo… Iceweasel), then you can try looking at the following.

### The Book of Mozilla

About:mozilla is oddly Biblical and has been changing in new versions along with Mozilla development since early 90s Netscape. It’s just really weird. The current version, as I write this, reads: “The twins of Mammon quarrelled. Their warring plunged the world into a new darkness, and the beast abhorred the darkness. So it began to move swiftly, and grew more powerful, and went forth and multiplied. And the beasts brought fire and light to the darkness.” The Book of Mozilla.

Go to any video on YouTube (it has to be on youtube; not embedded from youtube onto another site). Start playing a video (beyond the ads) and pause the video. With the video paused, hold down the left arrow button on your keyboard and (while holding it down) press the up button.

## CLISP

### Menorah Candles

The menorah has long been the logo for CLISP. This page explains why. When you run CLISP, you get the menorah, as in the following.

$clisp i i i i i i i ooooo o ooooooo ooooo ooooo I I I I I I I 8 8 8 8 8 o 8 8 I \ +' / I 8 8 8 8 8 8 \ -+-' / 8 8 8 ooooo 8oooo -__|__-' 8 8 8 8 8 | 8 o 8 8 o 8 8 ------+------ ooooo 8oooooo ooo8ooo ooooo 8 Welcome to GNU CLISP 2.49 (2010-07-07) <http://clisp.cons.org/> Copyright (c) Bruno Haible, Michael Stoll 1992, 1993 Copyright (c) Bruno Haible, Marcus Daniels 1994-1997 Copyright (c) Bruno Haible, Pierpaolo Bernardi, Sam Steingold 1998 Copyright (c) Bruno Haible, Sam Steingold 1999-2000 Copyright (c) Sam Steingold, Bruno Haible 2001-2010 Type :h and hit Enter for context help. [1]> What many CLISP users haven’t noticed is that the candles on the menorah are lit properly during Hanukkah. ## Vim ### Help with the meaning of life… In Vim, if you issue the command ‘:help 42′ you get the following output. What is the meaning of life, the universe and everything? *42* Douglas Adams, the only person who knew what this question really was about is now dead, unfortunately. So now you might wonder what the meaning of death is... ============================================================================== Next chapter: |usr_43.txt| Using filetypes Copyright: see |manual-copyright| vim:tw=78:ts=8:ft=help:norl: ## Apt-get and Aptitude The apt package management system (used by Debian-based Linux systems) has several Easter eggs, mostly related to cows. ### Cow Powers of Apt and Aptitude If you ask apt for help documentation, the last line that it returns is a bit weird. $ apt-get help apt 0.9.7.9 for amd64 compiled on Nov 16 2013 12:41:41 Usage: apt-get [options] command apt-get [options] install|remove pkg1 [pkg2 ...] ... ... ... -o=? Set an arbitrary configuration option, eg -o dir::cache=/tmp See the apt-get(8), sources.list(5) and apt.conf(5) manual pages for more information and options. This APT has Super Cow Powers.

So, we know that apt does have super cow powers. What does that mean? Well, it should probably be able to moo.

$apt-get moo (__) (oo) /------\/ / | || * /\---/\ ~~ ~~ ...."Have you mooed today?"... So, that works. What about aptitude? Since aptitude is supposed to be an improved apt interface, we’d expect something more. But, when we ask aptitude for its help string, we get the following. $ aptitude help aptitude 0.6.8.2 Usage: aptitude [-S fname] [-u|-i] aptitude [options] <action> ... Actions (if none is specified, aptitude will enter interactive mode): ... ... ... (terminal interface only) -i Perform an install run on startup. (terminal interface only)   This aptitude does not have Super Cow Powers.

Hmmm… so aptitude claims to not have super cow powers. What happens if we ask it to moo?

$aptitude moo There are no Easter Eggs in this program. OK. Maybe we just need to ask correctly… $ sudo aptitude moo [sudo] password for jason: There are no Easter Eggs in this program. $aptitude -v moo There really are no Easter Eggs in this program. Now, most people would just stop here. But, a real Easter egg isn’t found by stopping. What if we keep asking for more verbosity? $ aptitude -vv moo Didn't I already tell you that there are no Easter Eggs in this program? $aptitude -vvv moo Stop it!$ aptitude -vvvv moo Okay, okay, if I give you an Easter Egg, will you go away? $aptitude -vvvvv moo All right, you win. /----\ -------/ \ / \ / | -----------------/ --------\ ----------------------------------------------$ aptitude -vvvvvv moo What is it? It's an elephant being eaten by a snake, of course. \$ aptitude -vvvvvvv moo What is it? It's an elephant being eaten by a snake, of course.

That’s a good Easter egg.

### For those that misspell commands in a hurry

Some Linux distros have a command named ‘sl’ which is designed to mock those that misspell ‘ls’ at the command-line. (E.g., in Ubuntu/Mint/Debian you can use ‘sudo apt-get install sl’.) This command simply chugs a train across the screen, forcing you to wait and watch the train before you can attempt to enter ‘ls’ again.

## Irix

### Release note recipes

If you have an old SGI machine running Irix, type the following.

> relnotes dmedia_eoe 29

What should give you release notes … actually gives you a kung pao chicken recipe.

## Introduction

A while ago in this post, I proved that the sum of divisors of an integer can be computed as a multiplicative function on primes. Thus, knowing the prime factorization of an integer results in a direct algorithmic method for computing the sum of divisors of the integer without any combinatorial hassle.

This little trick, as well as many other factorization techniques (e.g., greatest common divisor of two integers), pops up quite a bit in the Project Euler problems. They pop up so often that I eventually found myself wanting to collect the routines in a Python module instead of copying and pasting the code every time I needed to do factorizations in Python.

So, today I pushed this little module. (There’s no setup script yet. Just place it in your working directory and ‘import primes’ for now.)

There are other ways we could go about this (using Sage to coordinate factorization methods, for instance). And, of course, for the more complicated problems (e.g., higher numbered Project Euler problems), such an approach in Python won’t do. But, for a surprising number of Project Euler problems below 150 or so, this module should help a ton.

## Factoring a Range of Integers

First, you should understand that there’s no elliptic curve method and no quadratic sieve. This module is designed to provide factorization tools (sum of divisors, gcd, lcm, etc.) for a large range or list of consecutive integers. While certain fancy methods are fast at factoring large random integers, they aren’t the best approach when factoring a big list of consecutive integers. A typical sieve and some careful planning works better in such a situation, as strange as that may feel.

## Examples: Prime Sieve Class

Using IPython, we’ll explore this module a bit and see what we can do with it. Then, we’ll show how it can be used to solve a Project Euler problem. For now, let’s start by creating a prime sieve. We’ll ask Python to compute all primes below one million.

In [1]: import primes   In [2]: time S = primes.Sieve(1000000) CPU times: user 0.12 s, sys: 0.02 s, total: 0.13 s Wall time: 0.12 s

How many primes are now included in this sieve? That is, how many primes are there below one million?

In [3]: S.numPrimes Out[3]: 78498

We can use the sieve to test any integer in the given range to see if it is a prime.

In [4]: S.isPrime(873) Out[4]: False

We can make a list of primes from the given sieve, in case we wish to use an iterable over primes.

In [5]: time L = S.listPrimes() CPU times: user 0.06 s, sys: 0.00 s, total: 0.06 s Wall time: 0.05 s

Now, L is a list containing all prime values less than one million. We can also make a set. The set will be slightly more expensive to create, but will provide faster membership access once created.

In [6]: time M = S.setPrimes() CPU times: user 0.09 s, sys: 0.01 s, total: 0.10 s Wall time: 0.07 s

If you forget what the limit point of the sieve is, it is contained as a variable inside the Sieve class.

In [7]: S.limit Out[7]: 1000000

## Examples: RangeFactor Class

The RangeFactor class uses a prime sieve to factor a list of integers. It then makes various methods available that use this collection of factorization records. Let’s start by factoring all natural numbers less than one million.

In [8]: time F = primes.RangeFactor(1000000) CPU times: user 2.13 s, sys: 0.06 s, total: 2.19 s Wall time: 2.18 s

We can get a more detailed time analysis by asking for verbose output.

In [9]: time F = primes.RangeFactor(1000000, verbose=True) primes : creating sieve primes : sieve created in 0.140506982803 seconds primes : creating factor records primes : factor records created in 0.232191085815 seconds primes : populating factor records primes : factor records filled in 1.8158788681 seconds CPU times: user 2.22 s, sys: 0.08 s, total: 2.29 s Wall time: 2.27 s

One of the most simple and obvious things to do now is to ask for the factorization of an integer in the given range. Currently, the factors are in a list with each integer having a dictionary of prime keys and exponent value pairs. This example should make that relationship a bit more clear.

In [10]: F.factors[5] Out[10]: {5: 1}   In [11]: F.factors[50] Out[11]: {2: 1, 5: 2}

The next most obvious thing we could do is to examine the greatest common divisor and least common multiple of two integers. We could also simply ask if two integers are coprime (which is slightly more efficient than computing the gcd or lcm).

In [12]: F.gcd(100,29754) Out[12]: 2   In [13]: F.gcd(100,6634) Out[13]: 2   In [14]: F.gcd(100,500) Out[14]: 100   In [15]: F.lcm(774,384) Out[15]: 49536   In [16]: F.areCoprime(774,384) Out[16]: False

Nothing here so far is too special. Really, we’re just making standard prime tools available more easily. But, now we can use the multiplicative function on primes to compute the sum of divisors of any integers in the given range. (Recall that a perfect number is a number that is the sum of all of its proper divisors.)

In [17]: F.sumDivisors(496) Out[17]: 992   In [18]: F.sumProperDivisors(496) Out[18]: 496   In [19]: F.isPerfect(496) Out[19]: True

Now, we can do some slightly more complicated computations. Let’s sum all the perfect numbers under one million. First, let’s see what they are. Then, we’ll take the sum.

In [20]: for i in range(2,1000000): ....: if F.isPerfect(i): print i ....: 6 28 496 8128

What’s a bit crazy here is that we’re computing the sum of divisors for ALL integers less than one million to check if each integer is perfect or not. All things considered, that’s going on pretty quickly due to our use of that nice multiplicative function.

In [28]: time sum([i for i in range(2,F.sieve.limit) if F.isPerfect(i)]) CPU times: user 1.60 s, sys: 0.03 s, total: 1.62 s Wall time: 1.58 s Out[28]: 8658

## A Sample Project Euler Problem

Problem 21 is stated as follows:

Let d(n) be defined as the sum of proper divisors of n (numbers less than n which divide evenly into n).
If d(a) = b and d(b) = a, where a ≠ b, then a and b are an amicable pair and each of a and b are called amicable numbers.

For example, the proper divisors of 220 are 1, 2, 4, 5, 10, 11, 20, 22, 44, 55 and 110; therefore d(220) = 284. The proper divisors of 284 are 1, 2, 4, 71 and 142; so d(284) = 220.

Evaluate the sum of all the amicable numbers under 10000.

## Python Solution

This is solved using the just discussed ‘primes’ module with the following code.

#!/usr/bin/python2   import time import primes   start = time.time()   # factor all numbers from 1 to 10000 F = primes.RangeFactor(10000)   # compute the sum of all proper divisors for integers from 1 to 10000 S = [F.sumProperDivisors(i) for i in range(10000)]   # find amicable pairs and add to a set pairs = set() for i in range(2,10000): if i in pairs: pass else: if S[i] < 10000 and S[i]!=i and S[S[i]]==i: pairs.add(i) pairs.add(S[i])   elapsed = time.time() - start   print "Found result %s in %s seconds" % (sum(pairs), elapsed)

When executed, we get the following output.

Found result 31626 in 0.0326828956604 seconds`