Ir al contenido principal

Frase

No es el conocimiento, sino el acto de aprendizaje; y no la posesión, sino el acto de llegar a ella, lo que concede el mayor disfrute

-- Carl Friedrich Gauss

El mundo se vuelve gris

Cuando uno va creciendo empieza a entender por que la gente mayor a veces sonríe pensando en sus recuerdos. Y es que a medida que vivimos y el pasar del tiempo lleva a la desaparición de gente que admirabamos o apreciabamos el presente se hace un poquito más gris, y el pasado se vuelve más brillante.

pull-right

Escribo esto porque en esta semana se fue uno de los tantos que yo admiro mucho, un humorista muy talentoso llamado Daniel Rabinovich parte del grupo cómico que jamás pude ver en vivo "Les Luthiers". Me arrepiento mucho de siempre haber pospuesto ir a verlos, ya que no va a ser lo mismo sin Daniel.

Aún así es bueno pensar en que tuvimos la suerte de ser contemporáneos a este grande, y ahora nosotros también podemos empezar a pensar en el pasado y sonreír un poco más.

Instalando Octave 4.0 en Fedora 22

Algo con lo que me topé usando Octave es que la mayoría de los entornos de desarrollo (incluso el plugin de Emacs) no funcionaban muy bien con la versión actual en los repos de mi distro. Entonces vi que la versión 4.0 trae GUI por defecto, pero no esta en los repos de Fedora. Por lo tanto lo mejor es instalarlo desde un repo Copr, para esto hacemos:

sudo dnf copr enable orion/octave
sudo dnf update octave 
#o dnf install octave

y tenemos Octave 4.0 con un GUI muy bonito: fit-width

Recapitulación de la semana

Esta última semana fue interesante, estuve trabajando y estudiando bastante.

Estudio

Después de toparme con algunas dificultades para entender algunos algoritmos de criptografía, (más que nada funciones generación de números primos pseudoaleatorios ) y también haberme topado con varios conceptos de cálculo estudiando Machine Learning que no entendía decidí hacer algo al respecto y retomar el estudio de Cálculo al menos tres veces por semana. Eso derivó (no pun intended) en aprender un poco de Octave (que también me servirá para Machine Learning) que me pareció genial ( en especial la versión 4.0 que ya viene con un GUI bastante lindo ).

Lectura

Ultimamente leí varias cosas, terminé de leer El Nombre de la rosa, el Señor de los anillos en inglés (de nuevo) y el comic Watchmen ( que es genial y amerita un futuro post ). Me prometí no comprarme libros nuevos ya que tengo pendiente terminar de leer la colección de Tolkien que me regalaron mi hermano y su esposa para mi cumpleaños, pero el sábado mientras paseaba con Eugenia me encontré el libro "Brave New World" de Huxley y recordé que era uno de los que tenía pendiente hace mucho, hasta ahora es fascinante.

Otras cosas

Nuevamente he estado algo obsesionado con los simuladores de vuelo, particularmente con el IL2 Sturmovik 1946 que es soprendentemente bastante mejor en realismo y personalización que Wings of Prey (que tengo desde hace unos años) a pesar de ser mucho más viejo.

También me encontre un muñeco de Tux (la mascota de Linux) que ahora supervisa mis sesiones de programación y estudio. No esperaba encontrar algo así en Paraná, al parecer es uno de esos muñecos de goma anti stress, pero no creo que lo apriete mucho ya que la pintura no parece de la mejor calidad y no quiero arruinarlo.

fit-width

Resaltado de sintaxis

Agregue otra funcionalidad al blog ahora resalta sintaxis gracias a prism.js

Ejemplo:

class Hola
    def initialize

        @texto="hola mundo"
        puts @texto
    end
end

Soy muy malo escribiendo con el celular

probando KaTeX

Mi escritorio de trabajo

Como para volver a escribir en el blog ( aunque sea un par de veces por semana) quería contar un poco sobre mi entorno de trabajo, aplicaciones y demás que utilizo a diario.

Window Manager

Estoy utilizando i3 que es un tiling window manager que me pareció genial, ya en su momento había utilizado ion3 que era un poco similar y Ratpoison pero nunca me convencieron. Una gran contra para mi de los tiling window manager es que algunas aplicaciones más orientadas al uso del mouse no funcionaban tan bien, y no es que tenga nada en contra de utilizar el mouse. Algo que me encanta de i3 es que podés ir organizando los escritorios al vuelo y no requiere un layout predefinido, separando en diversos contenedores horizontal/vertical, también poder cambiar los modos a pestañas o pila son geniales.

por ejemplo:

fit-width

Mi archivo de configuración esta un poco tuneado, pueden acceder aquí.

Ruby

Utilizo rbenv para manejar las versiones, luego de mucho pelear con RVM me pareció más entendible y un poco menos invasivo, hasta ahora funciona genial.

Editor de texto

Utilizo Emacs, sinceramente no he podido adaptarme nunca al uso de Vim (más allá de lo básico) los comandos me parecen más intuitivos y "recordables" (tengo pésima memoria). Ultimamente le he dedicado mucho a adaptarlo acorde a mis necesidades y las de mi trabajo ( estilo de código, lenguajes de programación utilizados, etc).

fit-width

Comparto mi .emacs

File Manager

A veces Caja (ex Nautilus), a veces Thuanr. Caja tiene más funcionalidades, Thunar es más ligero aunque esto no importa mucho en mi escritorio pero en mi laptop se nota más la diferencia

Algunos scripts

setwallpaper.sh es un script que te permite cambiar el wallpaper que estas utilizando, hace uso de fsetbg y zenity (para ser más amigable) , así que podes recuperarlo luego en el inicio de sesión con fbsetbg -l

mp4subtitles.sh esto es por una necesidad muy particular, a veces quiero ver videos con subtitulos en mi tele ( un Panasonic Viera) y por una de esas particularidades arbitrarias, solo se ven los subtitulos grandes si el formato es mkv con subtitulos embebidos ( en mp4 no se ven directamente y en divx + srt se ven chiquititos)

Curiosidad

Haciendo este post descubri que hay un paquete para emacs de gist, básicamente hay que hacer M-x package-install y luego gist, despues cada vez que querés subir el buffer actual a gist hacés gist-buffer y listo! Escribir ayuda a aprender :) .

The quest for a decent PHP request

WARNING: rant ahead

So, today I needed to put recaptcha on a PHP site, and I stumbled into the trivial ( for today's standards) problem of having to perform an HTTP query from PHP. If you are not familiar with PHP, traditionally this was done in two ugly ways. One was to use libcURL. Which is a library with a rather ugly syntax:

$ch = curl_init();

curl_setopt($ch,CURLOPT_URL, $your_url_goes_here);
curl_setopt($ch,CURLOPT_POST, count($form_data));
curl_setopt($ch,CURLOPT_RETURNTRANSFER, true);

curl_setopt($ch,CURLOPT_POSTFIELDS, http_build_query($form_data));
$result = curl_exec($ch);

curl_close($ch);

this is certainly ugly for many reasons, it's non intuitive, curl has many obscure options and the most sane default (which would be to return the response contents as an object or plaintext) is disabled by default (that's why RETURNTRANSFER is important).

The other approach is using file_get_contents()/file_put_contents() I'll let you figure out by yourself why this sucks and it's not a "code smell" it's a rotting corpse with an unholy stench of non-intuitive crap. Also you are dealing with a file ¿? not an actual HTTP Request, you don't have proper access to response codes.

// Create a stream
$opts = array(
  'http'=>array(
    'method'=>"GET",
    'header'=>"Accept-language: en\r\n" .
              "Cookie: foo=bar\r\n"
  )
);

$context = stream_context_create($opts);

// Open the file using the HTTP headers set above
$file = file_get_contents('http://www.example.com/', false, $context);

Enter HttpRequest

Of course you can use HttpRequest from the pecl_http extension. Yeah... a language that basically revolves around http requests doesn't have a proper HttpRequest implementation in it's STL, I'll give you a few minutes to try to swallow that fact. The use case is explained by the PHP manual as follows:

$r = new HttpRequest('http://example.com/form.php', HttpRequest::METH_POST);
$r->setOptions(array('cookies' => array('lang' => 'de')));
$r->addPostFields(array('user' => 'mike', 'pass' => 's3c|r3t'));
$r->addPostFile('image', 'profile.jpg', 'image/jpeg');
try {
    echo $r->send()->getBody();
} catch (HttpException $ex) {
    echo $ex;
}

Which is not too bad, except for the ugly constant name METH_POST, which I start to suspect is a reference to the drugs that many PHP developers have started using after dealing with this kind of thing. Except for one more problem, this is the implementation for an old as heck pecl_http extensions. Because now we have a better PHP and we have namespaces so we had to make it more fancier, modular whatever. Lurking I find that this is now the proper implementation of a HTTP POST Request :

$request = new http\Client\Request("POST",
    "http://localhost/post.php",
    ["Content-Type" => "application/x-www-form-urlencoded"]
);
$request->getBody()->append(new http\QueryString([
    "user" => "mike",
    "name" => "Michael Wallner"
]));

$client = new http\Client;
$client->enqueue($request)->send();

// ask for the response for this specific request
$response = $client->getResponse($request);
printf("-> %s\n", $response->getInfo());

Holy smokes! $request->getBody()->append() ?? what is this? are we dealing with a request or a JavaScript DOM manipulation function? Now we have to deal with an HTTP Client, a Request, a querystring object and a the same ugly curlopts wrapped in an almost equal ugly way

oh my god.. this is utter garbage.

For the sake of comparision this is how other languages do it:

Go is still verbose, it still works with a request + client, but is a lot more clear(example taken from here ):

import (
    "bytes"
    "fmt"
    "net/http"
)


func main() {
    url := "http://xxx/yyy"
    fmt.Println("URL:>", url)

    var query = []byte(`your query`)
    req, err := http.NewRequest("POST", url, bytes.NewBuffer(query))
    req.Header.Set("X-Custom-Header", "myvalue")
    req.Header.Set("Content-Type", "text/plain")

    client := &http.Client{}
    resp, err := client.Do(req)
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()

    fmt.Println("response Status:", resp.Status)
    fmt.Println("response Headers:", resp.Header)
    body, _ := ioutil.ReadAll(resp.Body)
    fmt.Println("response Body:", string(body))
}

Ruby... well there isn't much to see:

uri = URI('http://www.example.com/search.cgi')
res = Net::HTTP.post_form(uri, 'q' => ['ruby', 'perl'], 'max' => '50')
puts res.body if res.is_a?(Net::HTTPSuccess)

And even Javascript which I hate (taken from my blog file upload):

  var uploadedFiles = document.getElementById("js-upload-files");   
    var fd = new FormData();
fd.append("media",uploadedFiles.files[i]);
        var request = new XMLHttpRequest();
        request.open("POST", "/admin/media/create");
        request.send(fd);
            request.onreadystatechange = function(){
         if(request.readyState == 4 && request.status == 200){ 
... do stuf....

                        } 
             }

Conclusion

So I won't claim PHP is doooooooomed, like many people like to say. Nor I'll talk badly about any PHP project just because it's PHP like many people do. I still believe that a good programmer will be able to do great things with it, and lousy programmers will manage to do really ugly things on any language they touch.

The problem is that PHP is catching up from many years of stagnation, but in many cases is borrowing really ugly ways from other languages because it doesn't seem to know what it wants to be, every other language seems to be coherent in the way that they express themselves, yet PHP seems to find a way that is convoluted and extremely verbose to do pretty much the same thing; and this matters not because it doesn't allow the programmer to code well, but because it certainly discourages them to adopt the newer and supposedly better practices. After trying to figure out a proper way with the code samples above, it's highly probably that we just decide to say "screw it" and do the things in the old and ugly but familiar way, and that in my opinion, sucks.

A bunch of stuff

I've been quite busy but I'm still trying to learn a thing or two daily.

Termbox

Termbox-go is a go wrapper to work with the Termbox library which allows to make simple console-based applications. It's quite neat and easy/straightforward to do. I've been learning a thing or two with it. And it has been fun thing to do during the weekend, I also used it as a joke for a couple of friends parodying the "hacking time" segment of the movie Kung Fury, you can see it in this gist

example 1433166472_2015-06-01-103331_374x453_scrot.png

You could probably make things way more useful than this, but doing useless stuff is sometimes both fun and educative.

Sinatra

I've started building a small ruby app to scratch an itch in my company. Mainly management of client feedback and such. I'm using Sinatra+Datamapper because I want to finish it quickly, later on I might port it to Go if I have the time, but Ruby is good enough for now. This has taught me how to better organize a Sinatra project (since I don't want to cram all that functionality on a single module) and testing the framework actions.

Crypto

Cryptography course keeps being amazing, but I'm a little delayed with the course (which was to be expected). I'm not too confident to have enough time to take the final exam, but in the worst case scenario I might just take the second term course to reinforce my knowledge (and implementing all other crypto methods that are not part of the course homework). The last week homework consisted on integrity validation, which is a little less exciting than other algorithms but it was fun enough for me to do the assignment. But basically it involves generating a SHA256 sum of a file backwards progressively in blocks of 1024 bytes, I made use of the io.ReaderAt interface for this.

package sha256

import (
    "crypto/sha256"
    "io"

)

func HashBlock(block []byte) []byte {

    hash := sha256.New()
    hash.Reset()
    hash.Write(block)
    return hash.Sum(nil)
}

func HashFile(input io.ReaderAt, size int64) []byte {
    var current []byte = make([]byte, 1024)
    var previous []byte

    rest := size % 1024

    var last []byte = make([]byte, rest)
    _, err := input.ReadAt(last, size - rest)
    if err != nil {
        panic(err)
    }
    var offset int64 
    if rest != 0 {
        offset = size - rest - 1024 
        previous = make([]byte, 32)
        copy(previous, HashBlock(last))
    } else {
        offset = size - 1024    
    }


    for  offset >= 0 {
        _, err := input.ReadAt(current, offset)
        if err != nil {
            panic(err)
        }


        copy(previous, HashBlock(append(current, previous...)))

        offset -= 1024
    }

    return previous
}

Meetups

I'm a little bummed that there wasn't any meetups of coding groups nearby, I find them extremely motivating to keep learning, perhaps I should make some sort of compendium of what I've learned these months and present it as a lightning talk, that might make it more interesting. But I can't really decide which topic I like the most to talk about. Tough choices.

SWOT

This is a fun one, I've been introduced to SWOT Analysis by an episode of Silicon Valley by the infamous character Jared. I thaught it was silly at first but I gave a try to the method in a couple of situations and it comes in handy when having to take hard decisions. It's funny how sometimes some completely unrelated tool heard by pure coincidence can prove itself useful at times. It seems to be meant to work on single "yes or no" kind of situations, I wonder if there are similar tools to help making decisions on several alternatives.

example 1433166684_SWOT.jpg SWOT in action (keep in mind that showing your SWOT session to any of the parties involved might end up in really awkward or life-threatening situations)

Blog Actualizado (Markdown y uploads multimedia)

Le agregue al blog la capacidad de escribir con markdown (ya que el editor wysiwyg era horrendo) así como la capacidad de subir imagenes, archivos y videos, los comentarios también soportan markdown pero no ciertos elementos html por cuestiones de seguridad. Los posts ahora tienen 2 tipos de formato para conservar compatibilidad con los posts más antiguos, aunque el editor html esta deshabilitado en el admin.

Archivos multimedia

example 1431808837_go.png

Quote

Often people, especially computer engineers, focus on the machines. They think, "By doing this, the machine will run fast. By doing this, the machine will run more effectively. By doing this, the machine will something something something." They are focusing on machines. But in fact we need to focus on humans, on how humans care about doing programming or operating the application of the machines. We are the masters. They are the slaves. - Yukihiro Matsumoto

  • Esto
  • Es
  • Genial

Decent Code Blocks at last!

class OneTimePadEncoder
  def self.encode(message,key)
    xorpair(message,key)
   # (hex_message = message.unpack("H*").pop.hex ^ hex_key = key.unpack("H*").pop.hex ).to_s(16)
  end

  def self.decode(ct,key)
    xorpair(ct,key)
  end

  def self.xorpair(a,b)
    a = a.unpack("C*")
    b = b.unpack("C*")    
    result = []
    a.zip(b).each { |x| result.push x[0] ^ x[1] unless x[1] == nil }
    result.pack("C*")
  end  
end

como siempre pueden obtener el código en github

Block Ciphers

Disclaimer: These posts help mostly myself to assimilate what I'm learning about cryptography and should not be taken as a final word or academic truth. I encourage you to correct me if you find that I'm wrong.

So as I promised here is my entry on what I've learned lately about Block Ciphers and their mode of operation. Basically a Block ciphers allows us to encrypt a block of n bits of data data, there are many block cipher implementations, among the most popular ones are DES ( now deprecated) 3DES, AES.

AES particularly operates on blocks of 16 bytes ( 128 bits) with keys ranging from 128 to 256 bits. Therefore we need additional constructs to actually make the block cipher useful for larger pieces of data, there are several modes of operation of block ciphers and among the most popular ones there are CBC (Cipher Block Chained) and CTR or Counter mode. CBC basically works by creating a random Initialization Vector (IV) of 16 bytes and XORing it against a block of the same length on the plain text message, then it uses AES to Cypher the result with a chosen key. Subsequently we use the output of this output and XOR it against the next block of the plain text message. Best explained by this diagram: example 1431810452_601px-CBC_decryption.svg.pngexample 1431810440_Cbc_encryption.png

The IV is usually prepended to the message and it's retrieved when needed for decryption which is basically the same process but using the AES decryption algorithm first and then XOR it against the IV. Then we feed the next iteration using the cipher text from the previous step.

example 1431810511_Ctr_encryption.pngexample 1431810516_Ctr_decryption.png

There is also another mode of operation which is Counter mode, which instead of relying on the output of the encryption algorithm to cipher the next chunk of plain text it just appends or adds a unit to the Initialization Vector and XORs it with the plain text. It's worth mentioning that while on CBC we need to perform an AES decryption to retrieve the message, in CTR mode we will actually perform the encryption in both ways. What is interesting about CTR is that since we can easily calculate IV + 1 without needing any other additional processing we can make this process concurrent, and that's what makes it an interesting algorithm to implement with go.

Preparing:

In order to implement the algorithm we will use some libraries, and we will build some functions of our own, first is the aes package in the golang library. Then a convenience XOR function defined as:

package tools

// XORS 2 []byte
func XOR(a, b []byte) []byte {
    var result []byte = make([]byte, len(a))
    for i := 0; i < len(a); i++ {
        result[i] = a[i] ^ b[i]
    }
    return result
}

The function is pretty straightforward, it takes two bytes as parameters and performs a bitwise XOR on each one, it will fail if the arrays are of different sizes, but since we're supposed to work with 16 bytes it shouldn't matter.

We'd define the CTR struct with a factory method to set the key

package ctr

import (
    "bytes"
    "crypto/aes"
    "crypto/cipher"
    "crypto/rand"
    "github.com/juanfgs/hmwk_crypto/tools"

)

type CTR struct {
    Cipher cipher.Block
}

func New(key []byte) CTR {
    cipher, err := aes.NewCipher(key)
    if err != nil {
        panic(err)
    }
    return CTR{cipher}
}

and some convenience methods to cipher each block with AES:

func (this CTR) EncryptBlock(ct []byte) []byte {
    var pt []byte = make([]byte, aes.BlockSize)
    this.Cipher.Encrypt(pt, ct)

    return pt
}

This one is just for having a better legibility

func GrabBlock(full []byte, blockSize int, index int) []byte {
    return full[index : index+blockSize]
}

Then we implement the Encrypt method as follows:

func (this CTR) Encrypt(pt []byte) []byte {
    var iv []byte = make([]byte, aes.BlockSize) 
    _, err := rand.Read(iv) // We generate a random IV of 16 bytes

    if err != nil {
        panic(err)
    }
    CT := append([]byte{},iv...) // we append the IV to the cipher text, 

    max := len(pt) / aes.BlockSize  // we determine the maximum blocks of 16 bytes we can get from the plain text
    rest := len(pt) % aes.BlockSize  // we determine the length of the remaining bytes to get from the plaintext
    CTChan := make(chan []byte, max +1)  // we define a channel to send our CT blocks, of size max +1 to store all the bytes
    IVChan := make(chan byte, max +1) // we define a channel to send our modified IV 
    for i := 0; i <= max; i++ {

        var blockSize int 

        if i == max {
            blockSize = rest
        } else {
            blockSize = aes.BlockSize
        }

        block := GrabBlock(pt, blockSize, i * aes.BlockSize) // we grab a block from the plaintext

        go func( Block []byte) {

            CTChan <- tools.XOR(Block, this.EncryptBlock(iv)) //we cipher it against the iv and send it back to the CT channel
            IVChan <- iv[len(iv)-1] + 1 // we send the increased counter through the IV channel

        }( block )

        iv[len(iv)-1] = <- IVChan  // we receive and effectively increase the counter

    }
    go func(){
        close(CTChan)  // we close the ct channel
    }()


    for val := range CTChan{
        CT = append(CT,val...) // we append the values from the channel to the final CT
    }


    return bytes.TrimSuffix(CT, []byte{0}) // we remove any zeroed bytes at the end of the CT

}

The decryption method is pretty similar:

func (this CTR) Decrypt(ct []byte) []byte {
    var PT []byte
    iv := ct[:aes.BlockSize]  // we extract the IV from the beginning of the CT 

    CT := ct[aes.BlockSize:] // we slice the actual CT for convenience


    max := len(CT) / aes.BlockSize
    rest := len(CT) % aes.BlockSize
    PTChan := make(chan []byte, max +1)
    IVChan := make(chan byte, max +1)
// this code does pretty much the same

    for i := 0; i <= max; i++ {
        var blockSize int

        if i == max {
            blockSize = rest
        } else {
            blockSize = aes.BlockSize
        }

        block := GrabBlock(CT, blockSize, i * aes.BlockSize) // now we grab blocks from the CT instead of PT
        go func( Block []byte) {

            PTChan <- tools.XOR(Block, this.EncryptBlock(iv))  // we ENCRYPT (not decrypt) the iv and send the XOR back
            IVChan <- iv[len(iv)-1] + 1

        }( block )

        iv[len(iv)-1] = <- IVChan

    }

    go func(){
        close(PTChan)
    }()


    for val := range PTChan{
        PT = append(PT,val...)
    }

    return PT
}

And voilá, basically we have our CTR encryption system ready, you can find the full code and a partial CBC implementation on this github repo.

This was a very exciting thing to do and I learned a lot about cypto, golang channels and more. I hope you liked it!