jump to navigation

promees.es — a weekend’s experiment October 15, 2013

Posted by CK in Networking, Research, Software.
Tags: , ,
add a comment

There is this concept I have been thinking about for quite some time now, and this weekend I got around to kicking off its development. There are additional things I want to implement for it, but in its current form it is probably ok to publicize. So here’s the idea: People make promises constantly. “I’ll fix this for you”; “Sure, I’ll review your work”; “I’ll take you on a holiday”; you get the drill. People are judged by how good they are keeping their promises. Abiding to your promises is, in real life, one of the main reasons why others respect you.

Another thing happening in real life is that this respect is communicated in private conversations. But what would happen if people were able to advertise an index of their capability to hold on their promises? Would this be of value to their peers? This site is an experiment to figure out such questions, using Facebook as a platform.

promees.es

Feel free to give it a try at promees.es and get back to me with thoughts/comments you may have (keeping in mind that it was built in 3 days or so, so dont’ be harsh!).

Sunday 5:30am fun with XHR, XSS, HTTPS and (session) cookies September 8, 2013

Posted by CK in Software.
Tags: , , , , , ,
add a comment

Ok at this time of the day there’s not much energy left, but having spent the day trying to figure this out, I decided to take a few minutes and write about it.

The use case is as follows:

  1. Browser is on HTTP page, instructed to make XHR to foo.example.com over HTTPS to authenticate (end goal to receive session cookie)
  2. Browser continues making all other requests to bar.example.com over HTTP using this session cookie.

Seems like business as usual. Same 2nd-level domain, should just work. Not really. Steps to take if you find yourself in the situation:

You will find that due to XSS and browsers setting extra request headers, or maybe because you are using application/xml or application/json content type in a POST request, you will probably need to add CORS support to your backend: Browsers will issue a pre-flight request with the OPTIONS verb. You can add the following headers to your response to this OPTIONS request:

Access-Control-Allow-Origin: *,
Access-Control-Allow-Methods: GET, POST,
Access-Control-Allow-Headers: X-Requested-With

Check if you need to allow headers other than X-Requested-With, but usually this should work for the specific task. Also, you may want to set Access-Control-Max-Age too. These response headers will allow the browser to continue with the GET or POST to authenticate.

It was an unpleasant surprise to see that, although a session cookie was returned successfully after enabling CORS, the browser would ignore it in followup XHRs over HTTP. It turns out there are two more things you need to do:

  • Add withCredentials: true in your XHRs (not sure if only the one over HTTPS needs to have it or every one, including followup over HTTP, I used it everywhere for now)
  • Add Access-Control-Allow-Credentials: true in your CORS setup from above.

Bonus: If you are using Angular.js you will need to put withCredentials configuration in the $http config object for 1.0.x, otherwise you can set it permanently on $httpProvider defaults for v1.1.1+ — see here.

Dispatch with Scalatra on AsyncServlet January 15, 2013

Posted by CK in Software.
Tags: , , ,
3 comments

A basic example of using Dispatch with Scalatra on Jetty 8.x / AsyncServlet 3.0. It took me a lot of time to connect the dots how to use Dispatch asynchronously and actually process the results (not simply return them) and I could not find examples how to use Scalatra with AsyncServlet either. So here it is for anyone who could find it useful and save themselves a few –or more– hours.

import _root_.akka.dispatch._
import _root_.akka.actor._
import org.scalatra._
import org.scalatra.akka.AkkaSupport
import dispatch._

get("/asynctest") {

    // Get the AsyncContext -- or create one.
    val asctx = 
        if (request.isAsyncStarted) request.getAsyncContext 
        else request.startAsync
    // Execute the rest of the route in an Akka Future
    val result = Future {
        // Create a RequestBuilder to be used by Dispatch
        val url = host("slashdot.org") <:< Map(
                    ("Accept-Charset" -> "utf-8")
                )
        // Make the request, receive it as Bytes not to
        // disturb binary results. Http returns a Promise
        // to a result and execution moves along. More
        // information on the concepts is provided on
        // the Dispatch web site.
        val response = dispatch.Http(url OK as.Bytes)

        // Set up a handler for Dispatch' return. This
        // is bound on the Promise created earlier.
        response.onComplete { x =>
            // Get a handle to the response and its
            // output stream, we'll need this to write
            // results as soon as we have them.
            val res = asctx.getResponse
            val os = res.getOutputStream
            // 'x' is the response of the upstream web
            // service
            x match {
                // An error occurred, the exception is
                // sent to the client (of course you
                // would prefer to handle this otherwise
                // in real-world code)
                case Left(exc) => {
                    os.print(exc.getMessage)
                }
                // Response was received with a 200 HTTP
                // code, everything went fine. Data is
                // written to the output stream as bytes,
                // after having set the encoding to UTF-8.
                case Right(output) => {
                    res.setCharacterEncoding("utf-8")
                    os.write(output)
                }
            }
            // It is crucial to "complete" the Asynchronous
            // Context, otherwise no response will be sent
            // to the client.
            asctx.complete
        }
    }        
}

Big fat disclaimer: I’m fairly new to Scala/Scalatra (and the JVM for that matter), so if there are mistakes in this example –or improvements to make– let me know and I’ll correct them in the post.

Update: Ivan Porto Carrero of the Scalatra fame comments below on a much easier/concise way to achieve the same result (minor edits of my own in the code that follows but the point is the same). So it turns out you can call .complete directly on an Akka Promise, without a need to move the servlet’s asynchronous context around:

get("/async2") {
    import dispatch._
    import _root_.akka.dispatch.{Promise => AkkaPromise}

    val prom = AkkaPromise[String]()
    Http(host("slashdot.org") OK as.String) onComplete {
        case Left(ex) => println(ex.getMessage)
         case r => prom.complete(r)
    }
    prom.future
}

Function references in Scala October 17, 2011

Posted by CK in Software.
Tags: , ,
add a comment

I’m starting to learn Scala, and have some trouble coming to terms with its syntax. Maybe I’m spoiled by Python. In any case, here’s an example of things that I find annoying:


scala> def f(x:Int):Int = x
f: (x: Int)Int

scala> def g(f:Int=>Int, y:Int, b:Boolean):Int = if (b) g(f, y, false) else y
g: (f: (Int) => Int,y: Int,b: Boolean)Int

scala> def h=f
:6: error: missing arguments for method f in object $iw;
follow this method with `_' if you want to treat it as a partially applied function
def h=f
^
scala> def h=f _
h: (Int) => Int

So basically, although it is fine to use just f when defining function g, in the definition of function h it is necessary to postfix f with an underscore.

I haven’t read the language reference yet, and there may be good reasons to do it like this (although I can’t think of any right now), but the inconsistency is quite confusing.

My personal little “cloud” May 24, 2011

Posted by CK in Internet, Productivity, Software.
Tags: , ,
2 comments

Here it is, and it works perfect:

  • You will need a VPS or a home server. I’m using the wonderful, fan-less Shuttle XS35GT with a small SSD, as it is also my HTPC
  • A Linux distribution. I very highly recommend Mint 10 if you’re using the XS35GT so that you get a working audio + wireless and a stable XBMC, otherwise you may wish to use an LTS release like Ubuntu 10.04 or Mint 9.
  • An installation of eGroupware, to use its addressbook and calendar modules with its GroupDAV and SyncML synchronization facilities. You can easily install eGroupware using the deb repository provided on the project’s site.
  • Thunderbird, Lightning and the SOGo connector to support GroupDAV. Make sure you don’t use the “SOGo Lightning” extension; at least for me it didn’t work. Then subscribe Lightning to the eGroupware calendar and addressbook. Don’t bother with TODO items, unless a flat list is your thing.
  • A SyncML application to synchronize your phones. My Nokia E71 comes installed with one, while on an Android you can use the wonderful Synthesis client. Synthesis offers clients for additional platforms, but I only tried the one for Android.
  • The amazing Tracks application for GTD. It takes some effort to install, but it is totally worth it. You can also subscribe Lightning to various views of Tracks exported calendars. I’ve subscribed only to the one for due items, so that they appear with deadlines in my calendar. There are also two mobile applications to sync with Tracks, one for the iOS and one for the Android. Unfortunately the latter doesn’t work yet with Tracks 2.0, but it looks like it’s only a matter of time before it does.
  • …and, finally, Mindtouch Core (DekiWiki) as my data sink. There’s also a for-pay version, but I’m using the free/open-source one, which is fine. It’s also installed via a deb repository. I guess others may prefer some other platform, but for me Deki is perfect.

Then, your router set to post its address to DynDNS/No-IP or a similar service, and some CNAMEs in your domain to point to the hostname you have chosen (or simply the address of your VPS). All three services (Mindtouch, Tracks, eGroupware) are powered by Apache2, on virtual servers over HTTPS.

The data is yours!

PS: Special thanks to Yannis for suggesting to use eGroupware instead of SOGo+Funambol. A great improvement, indeed.
PPS: I only accept IaaS to fall under the term “cloud computing”, hence the quotes in the title.

Suspending a Dell 6410 w/ Mint 11RC May 20, 2011

Posted by CK in Software.
Tags: , , , ,
add a comment

If you just installed Linux Mind 11 RC on a Dell Latitude 6410 and, while on wireless, your system crashes when trying to suspend it, try this:

sudo mkdir /var/run/wpa_supplicant

This should fix it.

Blogilo January 15, 2011

Posted by CK in Software.
Tags:
add a comment

BTW, the previous post was the first one I wrote with Blogilo, and I like this tool quite a lot. Simple, and it just works.

Steve, I won January 15, 2011

Posted by CK in Mobility, Productivity, Software.
Tags: , , , ,
add a comment

After spending a considerable amount of time (yes, George, I know) looking at my options and trying various ways to synchronize desktop PIM data with my Nokia E71 phone, I eventually managed to make it work. It wasn’t piece of cake, and it requires running on my laptop some additional services, which I would not otherwise run. But it works. The main problem to deal with, is understanding your options. I tried many different setups with various combinations of Kontact / Evolution / Thunderbird+Lightning / Funambol / SOGo / OpenSync / SyncEvolution. Some of them appear to work for some people, but none worked for me in complete. Some that worked partially were

  1. Evolution + SyncEvolution + Funambol, but unfortunately Evolution was giving me so much pain re: my IMAP severs, that I just couldn’t stay with it;
  2. Thunderbird + SOGo connector + Funambol, but after creating TODOs and events I could not edit them anymore (this is a known bug, which remains unresolved).

Eventually, what worked for me was a combination of Kontact, Citadel, Groupware Sync server (customized Funambol release) and the Funambol SyncML client on the phone (although Nokia’s native client is used underneath and would apparently work directly if I tried it). This setup works almost out of the box (well that’s sort of a euphemism, admittedly); if you want to reproduce it, here’s how to do it:

First, install Citadel. This acts as a bridge between Kontact and Funambol, using GroupDAV on one side, and a Funambol connector on the other. Kontact is a full GroupDAV implementation, and so is Citadel. The alternative is eGroupware, but Citadel’s being fully open with no "upgraded" versions was the key factor to try it first. I never tried eGroupware eventually. The installation of Citadel (from source, as I could not find a Fedora package) was smooth and just happened. Configuration was, more or less, painless.

Then, I created a new KDE standard calendar resource from within System Settings, using GroupDAV and connecting to the local Citadel server. This bypasses akonadi, which has plenty of problems to solve, and is used immediately within Kontact (KOrganizer). Works like a charm. Following that, there was the biggest challenge: Going through akonadi for contacts. Unfortunately, KAddressbook cannot bypass akonadi; using a GroupDAV-based contact store must necessarily go through it. Until I managed to get it right, I had to fight with data store inconsistencies and delayed synchronization, disappearing contacts, and the like. Eventually it worked, when I created the new akonadi addressbook resource via KAddressbook and, before inserting any contacts, I set (via "Folder properties") an "Interval check time" of 2 min, "Local cache timeout" of 5 min. Apparently the exact values are semi-random and don’t play an important role, but it is (I guess) important to deactivate ""Inherit cache policy from parent". Based on the set up described above, contacts are always synchronized without problems, albeit with a delay of up to 5 minutes.

Having completed all that, it was time to install the Groupware Sync server, which was as easy as it gets. It knows where to find the local Citadel installation, and the built-in users are created automatically based on Citadel accounts. So not much more to do on this side either.

The last step was to install the Funambol SyncML client on the phone, and set it up. After some trial-and-error, I got it working. One of the things I had to do was to change the Funambol server’s port from 8080 to 80. As I am not running any other services on that port, it’s ok for me. I guess that eventually it would also work with 8080, if I would commit some more time to figure out the correct settings on the phone.

To make sure there’s a clean start, I removed all contacts and calendar entries from the phone — they were outdated anyway. The Funambol client has an option to do that very easily. Then I chose to synchronize everything, and, voila! My Kontact addressbook and calendar entries made it on the phone.

While using it I found out that addressbook entries are not synchronized when the default phone of the contact is a cell phone number (apparently a bug, it’s ok if it’s declared as a land line), and also that contact photos are not sync’ed (who cares). Perhaps these would be ok with eGroupware, but I’m not interested to change only because of that. Citadel’s appalling web interface? Yes, that could make me switch.

The result is much more important for me than simply synchronizing my PIM data. It means that, to a large extent, I am now safe from lock-in. I am running my PIM using an open integrated solution (Kontact) on an open desktop (KDE) and an open platform (Linux), so I’m safe enough on this side. Citadel is GPL as well, and Funambol is also open source (not sure about the exact license). All are running on my own infrastructure, and the data remains with me. On the phone side, I can use anything with a SyncML client available — and apparently, there’s one for most of the interesting phones/platforms out there. So I could simply switch to a different phone & OS, without caring too much about PIM synchronization. Which is, honestly, a blessing.

I’d love to hear if this article solved similar problems for you, so just leave a comment if you find it useful!

The story so far November 3, 2010

Posted by CK in IT, Productivity, Software.
Tags: , , , , , , , , , ,
1 comment so far

So here’s the summary of my linux@laptop adventures so far:

I started with Kubuntu, which as it turned out, after some kernel upgrade would not suspend to RAM/disk correctly. For a laptop, this is a no-go apparently, so after I realized that others also had the problem and that it would not be solved any time soon, I decided to give other distributions a chance.

I tried OpenSUSE, but after installation it wouldn’t even start. Without wasting too much time on this, I moved to Fedora 13; and it worked. It installed without problems, booted without problems, suspended without problems. After upgrading to the latest packages, I faced a common issue with newer Nouveau drivers, which wouldn’t work any more. By then, I had already found out about rpmfusion, which includes Nvidia drivers to install at the click of a mouse. Smooth.

Having solved the basics, I started using the system on a day-to-day basis. I thought I’d give a try to Gnome after a few years of faithfully discarding it, only to realize I was very much correct in doing so. Maybe Ubuntu has done a good job in its customizations, I don’t know, but the vanilla flavour in Fedora is ugly and unintuitive. Whoever disagrees, I would like them to walk into the shoes of a gnome-illiterate user and try to make changes such as setting date format to DD/MM/YYYY (instead of MM/DD/YYYY). I’m not interested in changing the whole system locale for that (and let’s forget about the fact that a linux apprentice knows nothing about locales). KDE, on the other hand, just works, and makes full sense when configuring and using it.

Where Gnome shines, is certain applications such as Evolution. It just rocks, especially when compared with Kmail. With the latter I had plenty of problems while using it with IMAP, but moving to disconnected IMAP was a game changer and Kmail now works quite well. In addition, Kmail failed to notify me while one of my IMAP accounts would not authenticate due to a server-side problem. The result was that for 4 days I would not get email there, thinking I was just not the recipient of any. This could have very bad consequences, for reasons irrelevant with this post. In any case, I would have already switched to Thunderbird or Evolution, but I want to have a desktop-wide addressbook that I can sync with a phone in the future, so I’m giving Kmail some more time and one more chance. In addition, Kontact is really nice in its entirety.

One more thing to mention in the “email” category, is spam detection. The default with Fedora/KDE/Kmail, is using SpamBayes, which would leave quite some spam in my mailbox even after some (admittedly, not too much) training. I then installed/tried SpamAssassin, but integration with Kmail was poor and spam would not be moved out of the mailbox even if marked as such. Eventually I went with Bogofilter and am happy to have done so, it works like a charm and improves a lot with training.

The, browser wars commenced. The default of Konqueror is slow and outdated in comparison to other browsers. I tried switching to the webkit kpart, which improved things a lot but didn’t solve many of the various problems such as random crashes. I really insisted, due to desktop integration, but at some point I just gave up. I made Firefox my default browser, and I’m very happy to have done so. Yesterday, I decided I can’t rely on Konqueror even as a second browser (I always keep 2 around). So for the first time, I decided to give Chrome a chance. So far I was resisting, mostly due to my concerns about Google. I must admit, the thing is *fast*. Although I haven’t switched to it as a main/default browser, I have been tempted to do so. In any case, it now serves as my alternative browser, should I need to test something without cleaning up cookies, or if Firefox does not work properly with some site.

Finally, when it comes to every-day usage, Office applications deserve a mention. Being realistic, I had to be able to run MS Office. I’m not interested in booting up virtual machines for this purpose, so I tried CrossOver for Linux. It works beautifully, and did not have a single problem so far. Well done.

PS: The Ubuntu font is amazing. It is my main desktop font.

Python memory management 101 July 12, 2010

Posted by CK in Software.
Tags: ,
add a comment

>>> a = 1
>>> b = a
>>> b += 1
>>> b
2
>>> a
1
>>>

…but:

>>> a = []
>>> b = a
>>> b.append(1)
>>> b
[1]
>>> a
[1]
>>>

Same for custom classes/objects.

Technically it makes sense, of course, to avoid deep-copying. On the other hand, it may be counter-intuitive in cases like this:

>>> a = 2 * [[]]
>>> a
[[], []]
>>> a[0].append(1)
>>> a
[[1], [1]]
>>>

It took me a minute to recover and realize, that statement a = 2 * [[]] does not create 2 different lists within “a”, but rather 2 references to the same list.