So I whipped up a function to toggle Wrangler on and off:

(setq erlang-refactor-status 0)
(defun toggle-erlang-refactor ()
  (interactive)
  (cond ((= erlang-refactor-status 0)
	 (call-interactively 'erlang-refactor-on)
	 (setq erlang-refactor-status 1))
	((= erlang-refactor-status 1)
	 (call-interactively 'erlang-refactor-off)
	 (setq erlang-refactor-status 0))))

I dropped this into distel_config.el and bound it to C-c C-r like so:

(global-set-key (kbd "C-c C-r") 'toggle-erlang-refactor)

Voila! Instant refactor mode toggle with two keystrokes.

I’m always interested in suggestions in how I can make the screencasts better. Send suggestions to kevin at (remove this) hypotheticalabs period com. They’re targeted towards someone completely new to the language, so bear that in mind when writing up suggestions.

Erlang R12 is unsupported yet on Windows and will only be supported with the 2.0.2 release.

The system uses a Comet based programming approach. From our knowledge the system uses Mochiweb as its web server to handle the massive number of long polling connections.

More details are available from Facebook engineering blog.

(Thanks, Bjarne Däcker, for producing the graphs).

I decided to introduce another token, ‘GAP’, to denote an empty line. Most likely, the scanner, in its current state, will not be able to handle empty lines with white space in them, etc, and the code is starting to look a bit confused. Oh well…

The toplevel rule for a function now becomes:



form -> function dot : '$1'.

form -> function 'GAP' : '$1'.



and the rule for alternative function clauses is as before:




function_clauses -> function_clause : ['$1'].
function_clauses ->
   function_clause ';' function_clauses : ['$1'|'$3'].
function_clauses ->
  function_clause 'OUT' : ['$1'].
function_clauses ->
  function_clause 'END' function_clauses : ['$1'|'$3'].




The first two rules are the original rules for indentation-insensitive code. The last two are for the indentation tokens. The ‘OUT’ token is for symmetry, to match the ‘IN’ token after the arrow in function_body. Remember that indentation tokens are normalized in the scanner.

The test program now looks like this:




-module(test).

-compile(export_all).
-scan(indentation).

f(X) ->
    X+2

g(X) ->
    X+4
.

h(X) ->
    Y = case X of
          a ->
            {a}
          b ->
            {b}
    end
    Y

i(a) -> a
i(b) -> b

test() ->
    2 = f(0),
    4 = f(2),
    4 = g(0),
    8 = g(4),
    {a} = h(a),
    {b} = h(b),
    a = i(a),
    b = i(b),
    ok.




A little bit better. The ‘end’ tokens are still needed, though. One thing at a time…

I made a copy of erl_scan.erl (named erl_scan_ind.erl) and made it figure out indentation tokens. Then I added to the Erlang grammar in erl_parse.yrl. All the old rules remain, but some new rules were added to account for indentation tokens. For example:



clause_body -> '->' exprs: '$2'



becomes:



clause_body -> '->' exprs: '$2'

clause_body -> '->' 'IN' exprs 'OUT' : '$3'.



The indentation tokens I used were:

• ‘IN’ for indent


• ‘OUT’ for outdent (one for each matching indent)


• ‘ALIGN’ for when the next line keeps the same indentation


• ‘END’ when indentation goes back to zero

So a sequence of expressions could be written without commas, based on the following rule:



exprs -> expr : ['$1'].

exprs -> expr ',' exprs : ['$1' | '$3'].

exprs -> expr 'ALIGN' exprs : ['$1' | '$3'].



My test program, which I was eventually able to compile, looked like this:




-module(test).

-compile(export_all).
-scan(indentation).

f(X) ->
    X+2
.

g(X) ->
    X+4
.

h(X) ->
    Y = case X of
          a ->
            {a}
          b ->
            {b}
    end
    Y
.




(Note especially that the final ‘end’ must be aligned with the Y, rather than the ‘case’. Perhaps this could be avoided…?)

The ending dots don’t have to be on their own line. Getting rid of them was too hard for me, since ‘dot’ is the end token for the Erlang grammar.

The -scan(indentation). attribute tells epp to switch to the indentation-sensitive scanner. -scan(normal). tells it to switch to the normal scanner.

I soon realised that I had to normalize the indentation tokens at the end of the scan. A few oddities were introduced, like inserting an ‘OUT’ token before each dot (and corresponding additions to the grammar). But for the most part, the additions to the grammar seemed fairly logical. The parser seems to handle all the old code, even though I should perhaps try recompiling the whole OTP source tree before making such a claim.

The code (based on OTP R12B-1) can be found at http://svn.ulf.wiger.net/indent/trunk

The grammar is still contaminated with some debug statements, which allowed me to print the productions as they were identified. They should of course be removed eventually.

I’m not convinced that this is really a good idea, but at least I had fun doing it.

In Response to “What Sucks About Erlang”. Yariv Sadan responds to Damien’s criticism.

New features include:

• Bidirectionnal protocols support


• Kernel polling and SMP support


• Loops support in scenarii


• And many other smaller improvements.

Tsung 1.2.2 includes near than 30 improvements over Tsung 1.2.1. A complete list of changes can be retrieved from Process-one support site.

The new code can be downloaded from Tsung download page.

3 Vote(s) ]]> http://www.planeterlang.org/out.php?title=Dead_Simple_Process_Pooling http://www.planeterlang.org/story.php?title=Dead_Simple_Process_Pooling Mon, 03 Mar 2008 14:11:35 CET blogs http://www.planeterlang.org/story.php?title=Dead_Simple_Process_Pooling The pool module provide a really simple, but useful, process pooling and load balancing mechanism. pool:start/1 is used to start worker Erlang nodes and pool:pspawn/3 or pool:pspawn_link/3 is used to queue work items into the pool.

The thing I really like about pool is it handles code loading transparently. Meaning that the worker nodes can access all the code the master node has available on its code path. This makes setting up a process pool even easier since there’s no per-worker configuration needed.

At the bottom of the post is the source for a Erlang module which uses pool to calculate fibnoacci numbers. Highly useful, I know. But sometimes the classic examples are the best A quick word of thanks to fellow Erlang Studio attendee Noah Thorp for his efficient fib implementation.

There are four easy steps to using the fibber module:

1. Configure the master node for disributed Erlang and setup the required code paths like so: erl -sname foo -cookie erlang -pa .
   


2. Call fibber:start/1 with the desired number of workers nodes.
   
   Example:
   
   

   
   (foo@perdido)1> fibber:start(3).
   Started worker3
   Started worker2
   Started worker1
   ok
   (foo@perdido)2>
   

   
   

   If you’re running a Unix-like operating system, you can use the following command to verify that master + # of workers Erlang processes are running: ps -A | grep beam | grep -v "grep beam" | wc -l
   



3. Start calculating fibonacci numbers by calling fibber:calc:
   

   
   (foo@perdido)2> {Node, Result} = fibber:calc(15).
   {worker1@perdido,610}
   (foo@perdido)3> Node.
   worker1@perdido
   (foo@perdido)4> Result.
   610
   

   
   


4. When you’re all done using fibber, call fibber:stop() to shutdown the process pool.

No other language I’ve ever used, including Java, Python, Ruby, or C++, has made thread/process pooling so easy or flexible.

-module(fibber).

-export([calc/1, calc/2, server_calc/2, start/1, stop/0]).

start(0) ->
	ok;

start(Workers) ->
	Name = list_to_atom("worker" ++ integer_to_list(Workers)),
	pool:start(Name),
	io:format("Started ~p~n", [Name]),
	start(Workers - 1).

stop() ->
	pool:stop().

calc(N) ->
	fibber:calc(N, 10000).

calc(N, Timeout) ->
	pool:pspawn(fibber, server_calc, [self(), N]),
	receive
		{reply, Result} ->
			Result;
		Wtf ->
			io:format("Didn't expect ~p~n", [Wtf])
	after Timeout ->
		timeout
	end.

server_calc(Caller, N) ->
	Result = fib(N),
	Caller ! {reply, {node(), Result}}.

fib(1) -> 1;
fib(2) -> 1;
fib(N) when N > 2 -> fib1(N,1,1).

fib1(3,P1,P2) -> P1 + P2;
fib1(N,P1,P2) ->
    fib1(N-1,P2, P1 + P2).

(See The full article for more details).

John Hughes

“Testing with QuickCheck” (Download) (Slides)

[See post to watch Flash video]

You talk about this book being less broad, but deeper than Programming Erlang: Software for a Concurrent World. What does this mean to you?

Francesco To me, it means the ability to ensure that everyone, regardless of background and previous knowledge of programming languages, is able to pick up the book and learn Erlang. Having taught Erlang to everyone, from corporate clients to university students, I know where they struggle and where the problems have been throughout the years. On one side, this resulted in our training materials evolving. On the other, it resulted in our teaching methods and hands on examples evolving. We are going into all of these details, explaining in depth areas we know readers will struggle with. We will be using many more examples, both from our training material and from our lectures, where we ask the delegates to use the shell as we explain the theory.

Why now? What makes you think the market is ready for another book on Erlang? What’s changed?

Francesco Erlang is expanding exponentially. Concurrency and SMP together with Joe’s book have pushed it to its next level. This has resulted in a critical mass which is helping us generate even more users. The market is ready not for one or two books, but for many books, as they generate traction and help each other. My hope is that this will result in an even greater critical mass.

Mike I tend to sign books on technologies that I want to learn about, or that I think I need to learn about. We tend to cover this up with lots of stuff that we call market analysis, but ultimately, that’s what it comes down to.

I was really surprised to see that the Haskell talks at OSCon were not just well attended, they had all the top spots. We didn’t have anything going on in Erlang, but my guess is that it would have shared the top of the list with Haskell. It’s not entirely clear what’s driving this. In the Ruby world, there are clearly a lot of concurrency issues that Erlang can help to solve. (One of the things that convinced me of the importance of Erlang was Lucas Carlson’s drproxy, which is a distributed reverse proxy server written in Erlang.

But I don’t think the Ruby community is really driving the interest in Erlang or in functional languages. There have been many other things happening: Microsoft’s F#, functional features in Sun’s Fortress, the Scala language, etc. What seems to be driving the interest is the idea that functional languages are going to make it easier to exploit parallelism, in a world where a low-end laptop has 8-16 cores. We’re not there now, but give it a few years and we will be. I certainly feel that programming paradigms are shifting, and they’re shifting in the direction of functional languages. Erlang is benefitting from that shift—particularly since Erlang was designed from the bottom up with concurrency in mind.

A detailed changelog is also available on ProcessOne bug tracker.

ejabberd 2.0.0 is available from ejabberd download page.

Thanks

A big thank you to everyone who helped making this version what it is
currently. This is a list extracted from ejabberd's ChangeLog since it
was started 5 years ago until 2.0.0:

AV, Andreas from Unstable.nl, Andrey Zamaraev, Andrzej Smyk, Andy
Turner, Anton Vanin, Asdx, Badlop, Balabanov Dmitry, Brian Campbell,
Carlos E. Lopez, Christophe Romain, Christophe Romain, Cord Beermann,
Doruk Fisek, EQHO Communications, Etan Reisner, Evgeniy Khramtsov,
Felipe Brito Vasconcellos, Geoff Cant, Iceburn, Igor Goryachev, Jacek
Konieczny, Jean-Sebastien Pedron, Jerome Sautret, Juraj Michalek,
Justin Karneges, Karl-Johan Karlsson, Konstantin Khomoutov, Leif
Johansson, Luca Brivio, Lucius Curado, Lukas Polivka alias Spike411,
Magnus Henoch, Marina Hahn, Marshall T. Rose, Martijn van Beers,
Martin Pokorny, Marvin Preuss, Massimiliano Mirra, Max Loparyev, Maxim
Ryazanov, Michael Scherer, Michael Shields, Mickael Remond, Miguel Gomard, Milos
Svasek, Nikolaus Polak, Oleg Palij, Oleg V. Motienko, Pablo Saratxaga,
Patrick Dreker, Pedro Melo, Pieter Rautenbach, Qu1j0t3, Renato Botelho, Robin Redeker, Ruslan
Rakhmanin, Samuel Tardieu, Sander Devrieze, Sergei Golovan, Shelley
Shyan, SkLUG, Stefan de Konink, Tomas Janousek, Tomasz Sterna, Tony
Finch, Torsten Werner, Tsukasa Hamano, Usercard, Vicent Alberola,
Victor Hugo dos Santos, Vincent Ricard, Vladimir Kulev, and Zbyszek
Zolkiewski.

• They get the idea that there are two groups of people wanting to learn Erlang (Erlang programmers and programmers who want to learn more about Erlang's approach to concurrency) and they are targeting both camps.


• They're talking about more than one book — I'm not sure how many though


• Their first book is planned to be very hands-on, which I think is great.

Once they're a little bit closer to making announcements, maybe I'll have some more news to post. On a related note, I'm hoping to have some more information on O'Reilly's Practical Erlang soon too.

Almost everyone in the class had little to no experience with Erlang so the first day was necessarily slow. As Dave said several times, the first day of class is the hardest to get right because you have to bring everyone up to roughly the same level. There were a few of us who had Erlang experience so the day was more of a review but Joe and Dave did an excellent job presenting the basics. I even managed to learn a few new things I either didn’t know or did know but should use more often:

• In general, smaller programs benefit more from HiPE than larger programs, according to Joe.


• Message sends w/acks are common pattern in Erlang used to avoid producer overrunning the consumer. Thinking of Erlang messages more like network packets and less like Smalltalk/Ruby messages is the more accurate mental model, I think.


• Patterns can be used to assert an invariant when deconstructing a list or tuple. For example, if I wanted a pattern which asserted the first and third values in a tuple were the same I could use the following pattern:
  

  {A, B, A} = {1, 2, 1}
  



• The head position of a list can be deconstructed in a single line of code. Instead of:
  
  [H|T] = [{name, "Kevin"}, {favorite_lang, "Erlang"}]
  
  

  I can write:
  
  [{name, Name} | T] = [{name, "Kevin"}, {favorite_lang, "Erlang}]
  
  I need to remember this more often. It’ll definitely a win in terms of lines of code written and it’s still comprehensible.
  



• In the Erlang shell, v() returns the value of a previously evaluated expression. v(-1), v(-2), etc returns the value for the previous, next previous, and so on values. v(22) returns the value for the expression evaluated at that Erlang prompt number. This is really useful when I forget to save a value into a variable.


• Joe gets TDD, or at least appears to. We’re writing failing tests before writing working code in class


• term_to_binary, binary_to_term — I need to prototype using these functions. A nice way to sling code around between Erlang nodes

After class a bunch of us met up with Martin Logan and a few other Chicago Erlangers (sorry, I didn’t catch everyone’s names) for drinks and food at the Blue Line Club Car. Accommodations were a bit tight in the restaurant but it was great to sit around a few drinks and talk about Erlang with other enthusiasts. I’m really intrigued by Martin’s latest project, Erlware as I think it addresses a lot of the problems I see with CEAN. Erlang definitely has a RubyForge-shaped hole in it and I think Erlware might be a good way to fill it. I’ll definitely give it a spin once the Studio is over and I’m back home and rested.

[Summary of Day 2]

Experience in Erlang and/or XMPP protocol is a big plus, but is not mandatory.

The jobs are preferably based in France, but our company being international and our team distributed across the world, remote work is possible.

If you are interested, please drop me a mail (mremond at this website domain) with a summary of your technical achievements and a description of why you would love working at Process-One.

1. the authors are also involved in teaching Erlang classes


2. there is going to be a website for the book soon


3. the book is going to be deeper (though less broad) than Programming Erlang: Software for a Concurrent World

I’m hoping to get an interview with the authors as the book gets closer to publication. Until then, I’ll see what more information I can lay my hands on.

“The runtime system with SMP support can now use multiple, thread specific instances of most memory allocators. This improves performance since it reduces lock contention in the memory allocators. It may however increase memory usage for some applications. The runtime system with SMP support will by default enable this feature on most allocators. The amount of instances used can be configured.”

So SMP/multi-core machines should see a nice speed up due to more efficient memory allocation and garbage collection.

So what are you waiting around for?! Download it now and discover the pleasure of easy concurrency!

Oh, and if you need help figuring out how to build it on your OS X box, might I suggest my screencast on the very topic?

What I've really been having fun with is mnesia, the native data storage application. With it you can do some really powerful things like advanced data distribution, federation, complex queriers, full/compartmental replication and the list goes on and on.

However, there doesn’t seem to be any authoritative definition of the term “Erlang-Style Concurrency”. I thought I’d at least try to give my view.

Joe Armstrong writes the following in his book, Programming Erlang:

“In Erlang:

• Creating and destroying processes is very fast.


• Sending messages between processes is very fast.


• Processess behave the same way on all operating systems.


• We can have very large numbers of processes.


• Processes share no memory and are completely independent.


• The only way for processes to interact is through message passing.”

I guess this is about as concise and authoritative definition as any. However, it’s not the whole story.

I would add the following items:

• Message passing is asynchronous.


• Processes can monitor each other.


• It is possible to selectively receive messages.


• Remote processes appear largely the same as local processes.

So this roughly describes how concurrency works in Erlang. Are all these bullets necessary for Erlang-style Concurrency? Perhaps not.

I would pick the following items as necessary for Erlang-style Concurrency:

• Fast process creation/destruction


• Ability to support >> 10 000 concurrent processes with largely unchanged characteristics.


• Fast asynchronous message passing.


• Copying message-passing semantics (share-nothing concurrency).


• Process monitoring.


• Selective message reception.

A few words to explain the above:

Speed and scalability

In order for concurrency to be useful as a fundamental modelling paradigm, the programmer must feel that processes are cheap enough you can efficiently create as many processes as the problem calls for. If there is any single defining characteristic of Erlang-style Concurrency, it is that you should be able to model your application after the natural concurrency patterns present in your problem. If creating a process is perceived as expensive, programmers will reuse existing processes instead; if message passing is perceived to be expensive, various techniques will be invented to avoid sending messages. These workarounds are nearly always detrimental.

There will always be limits. Erlang was designed for agent-style concurrency; not for massive data parallelism. You can “only” have ca 120 million concurrent processes in Erlang, provided you have enough memory. Personally, I’ve verified that Erlang performs consistently up to 20 million concurrent processes (I didn’t have enough memory to go further.) The cost of creating an Erlang process (ca 5-10 us) will be prohibitive for some applications, but it’s at least an order of magnitude cheaper than creating UNIX processes or POSIX threads. It’s reasonable to expect Erlang-style Concurrency to use roughly the same process granularity as Erlang does.

Asynchronous message passing

It has been argued that synchronous message passing is a better building block than asynchronous message passing. It is probably true that synchronous message passing is much easier to reason about, but asynchronous communication (”send and pray”) feels more intuitive in a distributed environment (in which case environments based on synchronous message passing will resort to some form of asynchronous communication as well). Anyway, it is reasonable to argue that asynchronous communication is a defining characteristic of Erlang-style Concurrency.

Copying semantics

Note that this doesn’t necessarily mean that messages must be copied, but they must act as if they were. There are several reasons why this is important:

• For reliability reasons, processes must not share memory


• In the distributed case, copying is inevitable, and we want as similar semantics as possible between local and distributed message passing.

It is perhaps too strong to require that processes share nothing, as this would rule out Erlang-style concurrency in most existing languages (even languages like Scala make it possible to share data between processes). Let’s just say that the Erlang style is to make it easier to use share-nothing concurrency than to use sharing.

Process monitoring

This is the enabler of “out-of-band” error handling, or the “let it crash” philosophy, which is very typical for Erlang. The idea is that you program for the correct case, and rely on supervising processes for error handling. This has turned out to be a beautiful way of handling fault tolerance in large systems.

Selective message reception

There are many ways to achieve selective message reception, but for complex multi-way concurrency, you have to support at least one of them. This is something often overlooked when approaching concurrency. It’s not obviously needed in simple programs, but when it is needed, you will suffer complexity explosion if you don’t have it.

I would suggest that ability to support selective message reception is sufficient for Erlang-style concurrency. It can be done either by matching on a single message queue (Erlang, OSE Delta), or by using different mailboxes/channels (Haskell, .NET, UNIX sockets). Pick one, and find a suitable example to experiment with. You may use my presentation “Structured Network Programming” (also attached as pdf) as inspiration, if you like.

Introduction

Since I designed the first version of the pluggable pubsub module for ejabberd in early 2007, I had in mind to turn it into a powerful application server. I have already blogged about the power of the new API of ejabberd 2.0 pubsub engine. However, this single article does not do any justice to this idea and how it can change the face of the web.

Customizable services based on pubsub is XMPP (eXtensible Messaging and Presence Protocol) at its full speed. The XMPP protocol has been designed since the beginning to be a near real time routing / distribution engine. A piece in the puzzle was missing however. This is where I think our new plugin based pubsub API fills the gap and turn ejabberd into the first XMPP application server.

As a proof of concept and as a way to robustify our API, we have written the Personal Eventing via Pubsub (PEP) as a plugin of our pubsub engine. It is a big specification and should clearly show the potential of the framework.

The XMPP application server: The Twitter case

The PEP implementation was still not enough to demonstrate that ejabberd is a fault-tolerant, highly scalable application development platform for the Web 2.0 era. I decided to start a serie of articles demonstrating how to build a distributed Twitter-like microblogging platform based on ejabberd 2.0.

Step 1

The goals of the first development step are the following:

• Write a new ejabberd service as a pubsub extension: this is a XMPP Twitter-like application code itself.


• Show that you can integrate with other pieces of the XMPP framework and not only the event distribution: The code uses the service browser as a user interface, building a fully dynamic hierarchy.


• Write the near realtime publishing / distribution code. To show how powerfull it is, I wrote the code in a way that node creation and subscriptions are not necessary. You only have to publish content and it is distributed to anyone that can see your presence. The end result, like PEP, is closely integrated to the user roster.

The architecture is distributed across as many user base (domain) as you want: A user can get events from users having their account in a large number of different servers.

Writing a new plugin to define custom node hierarchy

I wanted to build a special service hierarchy so that the application integrates nicely in XMPP service discovery. I wanted a dynamic discovery tree whose content was depending on who was doing the query. This would make the application more user friendly and the example much more impressive.

To write a custom node hierarchy with our API, you simply have to write a nodetree plugin. I have written the nodetree_twitter module to handle our special hierachy looking like the following:

• Root node: "twitter"


• Special node: MyStream. This is a collection displaying your latest published posts.


• Special node: FriendsStream. It contains the latest posts of my friends.


• Special node: MyFriends. It contains the list of your friends and allow you to see the latest posts of a given contact.

Friends are taken from the instant messaging contacts list: I defined this friends list as "the people that already share their presence with me". This way users do not have to manage the cumbersome subscription process specially for this application: Their friends from their contact list are cleverly reused.

Note that the node hierarchy is not defined in the Pubsub specification (XEP-0060), but is totally needed to make the application flexible. This pubsub node hierarchy does not violate the Pubsub specification, so we are fine with it.

This is the result as shown by Psi's service discovery::

The contact Blaine (thank you for your comments, by the way) is coming directly from my roster.

Posting entries

Posting an event is simply sending a small piece of XML code in ejabberd. As Psi does not support publishing events to our special virtual nodes (yet ), we will send it from Psi XML console.

For example, I can send the following piece of code in the XML console. I publish the post on my special pubsub node MyStream:

<iq type='set'
    to='pubsub.localhost'
    id='publish1'>
   <pubsub xmlns='http://jabber.org/protocol/pubsub'>
     <publish node='/twitter/MyStream'>
      <item>Demoing my Twitter-like code</item>
    </publish>
  </pubsub>
</iq>

The message is received instantly by all the contacts that have subscribed to my presence. No polling mechanism is needed. This is the beauty of using ejabberd XMPP App Server.

The received message is the following:

<message from="pubsub.localhost" to="test1@localhost" >
  <event xmlns="http://jabber.org/protocol/pubsub#event">
    <items node="/twitter/FriendsStream" >
      <item id="4B2C8277256F" >Demoing my Twitter-like code</item>
    </items>
  </event>
</message>

Note that the dynamic hierarchy is very powerful: We publish on our special node. The message seems to come from another special node on the other end, for the sake of consistency.

Persistence

I did nothing special for persistence of the published items. It is handled as a default by the ejabberd pubsub framework.

We can browse our new service to check that the blog post has been actually published:

I did not yet write the part of the code to browse friends posts. This will be part of a second article.

Conclusions

The end result is the basis of a Twitter-like application. It is fully distributed at the web scale. You can run this application on different servers and get events from people with account on different servers. It is compliant with ejabberd clustering and high scalability features. Most of all, thanks to our Pubsub API, I wrote this first code in 3 hours. It tooks something like 200 to 300 lines of code to write this extremely powerful infrastructure. Yes, that's amazing, I agree, as I was surprised myself by how easy and fast it was to write !

After a bit of clean-up, the code will be added to ProcessOne contributions Subversion repository. If you are impatient to play with the code, please drop me a mail

In a next article, I will present the second stage of this application. I will add better and complete browsing of the items and publish directly for the service browser thank to adhoc commands supported in Psi.

In the meantime, I hope to get this experiment running with client developers: It would be very nice see the messages received properly displayed in OneTeam and in Psi. Stay tuned !

The AOL XMPP scalability challenge

It has been largely covered on the Internet, that AOL is experimenting an XMPP gateway that allows users to connect to AIM and ICQ with a compliant XMPP (eXtensible Messaging and Presence Protocol) client.

This is still an early experience, but it proves that the interest in XMPP is growing at a tremendous rate. Big players like IBM, Google, Sun, Apple, Adobe and now possibly AOL are embrassing the protocol and it seems that the battle between XMPP and SIMPLE (Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions) is drawing to an end. The market has already choosen XMPP because it is closer to internet technology. SIMPLE feel like overly complicated Telco protocol. SIMPLE covers only a fraction of the scope of XMPP. XMPP supports a very large number of extensions.

The other interesting thing is that XMPP can win the battle over SIMPLE, but also over other proprietary protocol. Google choose it for Google Talk. AOL is considering it. The Instant Messaging is moving and 2008 will probably be a very intereting year for XMPP.

For us, XMPP protocol designers and XMPP server developers, this year might be one of the most challenging year. AOL is a very large IM domain on its own. We have tens of customers deployments that prove our XMPP server can scale to large number of users. Google Talk is proving too that XMPP can work on large scale. However, the next challenge is XMPP federation on a large scale.

Federation in the XMPP world means putting together several different users base so that they can share presence information and exchange message with others users, no matter on which server (called domain) they have created their account. In practice, the couple of servers that need to put users in touch, connect together and route messages.

Google Talk is federated with quite a good number of other XMPP servers as Jabber.org (powered by our XMPP implementation, ejabberd, and supported by ProcessOne) is. They must probably have the same number of connections, so around 3000 server connections for both is a good guess.

With the rapid spread of XMPP, two challenges need to be addressed:

Challenge 1: Massive number of federated servers

AOL experience will probably boost the number of XMPP server in production. 3000 connections to servers no Jabber.org is still very small. This figure is for one most famous XMPP server so it is likely that it is the largest number of server-to-server connexion in production.

What will happen when this number will grow ? Netcraft counted 70 millions of active domains in december 2007. This is a possible target for XMPP when all domains will have web, mail and Instant Messaging.

I have been thinking a lot about this problem lately. The new version of ejabberd, ejabberd 2.0, is a good step in this direction. ejabberd 2.0 has put a lot of effort in imporving the management of server-to-server connection. It is clusterisable and rely on special technical features to handle tens of thousand of connections on a single node. Connections between servers in ejabberd are now multiplexed to cope with large traffic between servers. We have also added protection mechanisms to detect when another server stall and deconnect it gracefully.

As a final note on this topic, my personnal view, however, is that we will need at some point in the future of XMPP to add meta routing nodes to avoid possibly 70 millions of servers to connect to 70 millions of other servers. We will share more thought on this point later on.

Massive number of users on two federated servers

This will exercice another possible scale limit: What will happen when two servers (for example AIM and Google) need to connect on each others for any reason, for example if one of the server need to restart ?

If those two servers have a large overlap in their users base, that is to say if many users from one servers are linked to many users in the other, those two server would need to synchronize presence for millions of users at once.

Again, we think we are leading the way in ejabberd with experimental code. We are experimenting priority mecanisms to synchronize presence in an incremental way when servers need to get in touch for the first time. We are also trying protocol improvement to limit the number of presence synchronization packets that need to be exchanged between two servers. If this works well we will propose this optimisation as extensions to the XMPP protocol.

On the same topic, another important feature when you select an XMPP server is fault-tolerance and high-availability. Those features are not only important because you cannot schedule downtime when your users are spread all over the world, but also because, each time you need to restart your server, you generate a possibly costly presence resync with the servers your users are in relation with (if you enabled federation of course). Our Instant Messaging solution provide features to upgrade the code in a running system. It means less maintenance downtime and thus less synchronization burden over the whole XMPP network when you restart one server.

Note that AOL experiment is only a gateway targeting only XMPP client use. We have no clue if AOL will allow communication with other XMPP servers in the future. This question is only purely theorical for the moment, but we need to make sure we are ready when it will become really important.

Conclusion

The big public servers are considering switching to XMPP. This is a great opportunity for the XMPP protocol and brings new challenge. As a XMPP server provider we think we are ready to face it. Let's hope 2008 will give us even more opportunity to prove it.

Other interesting read

On the scalability topic, you can read a previous blog post: Web 2.0: Shifting from "Get Fast" to "Get Massive"

This version is ready for testing for adventurous users that would like to help us making ejabberd the best release ever.

A detailled changelog is also available on ProcessOne bug tracker.

ejabberd 2.0.0 release candidate is available from ejabberd download page.

A big thank you to everyone who helped making this version what it is
currently. This is a list extracted from ejabberd's ChangeLog since it
was started 5 years ago until 2.0.0-rc1:

AV, Andreas from Unstable.nl, Andrey Zamaraev, Andrzej Smyk, Andy
Turner, Anton Vanin, Asdx, Badlop, Balabanov Dmitry, Brian Campbell,
Carlos E. Lopez, Christophe Romain, Christophe Romain, Cord Beermann,
Doruk Fisek, EQHO Communications, Etan Reisner, Evgeniy Khramtsov,
Felipe Brito Vasconcellos, Geoff Cant, Iceburn, Igor Goryachev, Jacek
Konieczny, Jean-Sebastien Pedron, Jerome Sautret, Juraj Michalek,
Justin Karneges, Karl-Johan Karlsson, Konstantin Khomoutov, Leif
Johansson, Luca Brivio, Lucius Curado, Lukas Polivka alias Spike411,
Magnus Henoch, Marina Hahn, Marshall T. Rose, Martijn van Beers,
Martin Pokorny, Marvin Preuss, Massimiliano Mirra, Max Loparyev, Maxim
Ryazanov, Michael Scherer, Michael Shields, Mickael Remond, Miguel Gomard, Milos
Svasek, Nikolaus Polak, Oleg Palij, Oleg V. Motienko, Pablo Saratxaga,
Patrick Dreker, Pedro Melo, Pieter Rautenbach, Qu1j0t3, Renato Botelho, Robin Redeker, Ruslan
Rakhmanin, Samuel Tardieu, Sander Devrieze, Sergei Golovan, Shelley
Shyan, SkLUG, Stefan de Konink, Tomas Janousek, Tomasz Sterna, Tony
Finch, Torsten Werner, Tsukasa Hamano, Usercard, Vicent Alberola,
Victor Hugo dos Santos, Vincent Ricard, Vladimir Kulev, and Zbyszek
Zolkiewski.

The idea of the Erlang Internet Framework, or EIF, was created at the beginning of 2007 therefore it is fitting to do a status update and ponder on the future of the project one year after it initial conception.

The EIF was originally created to be a way to combine all of the projects that I had been working on into one focused project, since then it has grown and several projects have been added. The original projects beyond the EIF were ErlMail, ErlWeb and ErlDir. Each of these projects are still in early development stages and they have been joined by ErlVoIP, a Voice over IP server, and ErlMedia, a Flash Media Server written in Erlang.

ErlMedia/ErlyVideo

The code base for my flash media server, which I was calling ErlMedia has been joined with ErlyVideo, which is another Flash Media Server written in Erlang. Since my initial work on the ErlMedia/ErlyVideo in August and September I haven’t been able to spend much more time on the project and the other team members have done a wonderful job of mending the problems that were plaguing the code I was working with and taking it much further then I could have done alone.

I am very hopefully that I will be able to spend a decent amount of time working on this project in early 2008, after the next release of ErlMail.

ErlMail

ErlMail has been the focus of my attention for most of 2007. The distinct lack of a complete email package, including library modules for both client and server, has been my inspiration for this projects since it’s inception. In my personal opinion, ErlMail has the most advanced email modules of anything currently available as open source in Erlang.

For the past several months I have been working diligently on the IMAP server in ErlMail. I knew this server was going to be difficult to create, but I had severely underestimated the amount of time it was going to take to make an IMAP server operational. I am down to the last command, the FETCH command, in the IMAP server before I release a stable version of it. It is not going to be RFC compliant in the next version and there are still a few commands that I will need to include after the next version as well, most notably the search command.

The feature that I think is the most notable in the ErlMail package is the modular message store. The modular message store allows users to create their own message store modules and simply change the user configuration file for ErlMail to use them. This feature already garners more interest then any other single feature in the ErlMail package and with my future directions, which you will read later, it will become even more important.

ErlWeb

ErlWeb, which is a web server and associated modules, got one official release in 2007. The most notable piece of this release is the tag based scripting language called Erlang Markup Language, or ERML. My intention with ERML is to create a way that people who do not know how to program in Erlang can still get the benifits of Erlang. The most recent shift in the design of ERML has been to add a way to extend ERML with user create modules that are not part of the ErlWeb package. This is being done now with the other packages in the EIF.

For instance, a module in ErlMail called erml_erlmail will have all ERML commands that are associated with email commands for use in the ErlWeb application. This allows for users to create their own ERML commands, written in Erlang, and include them into ErlWeb without recompiling the system. In the case of ErlMail, this also allows all of the email logic to be contained in one application while still interacting with the ErlWeb application. This feature appeared in the code base in December of 2007 and I consider it as important to ErlWeb as the modular message store is to ErlMail.

Similar files have been created for other EIF projects; such as erml_erldir and erml_erlvoip which are located in their respective applications.

Other Projects

The rest of the projects included in the EIF have not moved forward much in 2007. ErlDir did get an official release which is mostly library modules that handle packet encoding/decoding and compression/decompression and ErlVoIP has been announced although work on this project has not begun yet.

Google Code

During the second half of 2007 I moved all open source projects that I have onto Google Code. While this was a move on my part to allow me to be more portable with my development environment and create a pseudo backup system to ease my own mind this also allows for other people to join the projects and contribute their own code if they wish. I have one person who is helping me document (and correct my spelling errors) most of the projects already and I am open to the idea of other people joining the work to see these projects move forward faster then I could accomplish alone.

Future of EIF

I see a bright future for the EIF in 2008; starting with a new release of ErlMail in the first few weeks of the year. After that there are a few changes to the core idea of the system that I intend to implement. These changes reflect new innovations that were not around during the EIF initial conception and large needs that have arisen since then as well.

The EIF Director

I’m not sure I have ever described the main project for the EIF itself. The EIF project is intended to be a manager application that keeps track of the versions of the other applications and which nodes have what servers running. So while each application in the EIF will be independent, the EIF application will be the glue that holds them all together.

Some of the features that are planned for the future of the EIF application itself are:

• Upgrading itself and other EIF applications


• Monitoring which nodes are running what servers


• Load balancing traffic across all nodes


• Fail-over on node crashes; allowing a user connection to move from one node to another without the user noticing

The EIF application would handle much of the communications and some of the state data for each connection, and then the server processes on each node would do the work that each connection is requesting. The design goal of this is to allow nodes to be added or removed into the system without users noticing any interruptions. This would include the ability to move a long session that a user has, on say an IMAP server, from one node to another without the user needing to reconnect. I am also hoping to implement this into ErlWeb so that comet connections could be moved from one node to another without the user noticing that a node has been removed from the cluster.

Amazon EC2/S3/SimpleDB integration

The Amazon web services, or AWS, announcements, that have been on going for several months, have been unexpected and potentially a huge boost to the design on the EIF. I will be taking some time to look at these services and creating a road map to completely integrate these services into the EIF at every level that is reasonable.

The first use of the AWS will be in ErlMail. I intend to create a modular message store that incorporates the AWS as the backend, most likely using the SimpleDB for user and domain information and the S3 storage for the actual messages.

After the ErlMail message store is integrated with the AWS all other project will be designed to work with the EIF Director on EC2. With the flexibility of the EC2 cloud any number of Amazon Machine Images, or AMIs, could be made with different server configurations. Then as the EIF director noticed a burst in traffic or number of connections the EIF director could initiate a new instance of an EC2 AMI. When the load has lowered enough the EIF director could move sessions onto the oldest running instances (without users noticing) and disconnect from instances that are no longer needed.

The core configuration of the EIF on AWS could start out as one instance with the EIF director and all other servers on the same instance and as load increases the EIF director will expand or contract the needed nodes. As more nodes are added the EIF director will move session that are active on itself onto other nodes allowing the EIF director to focus it’s resources on controlling traffic flow and balancing connections. Eventually as load decreases the EIF director would take the sessions back onto itself to minimize the number of instances back down to one instance.

If implemented well this could create a system where no matter what type of services you want to provide you would be able to use the AWS to create the project and if the need arises the project could be moved to a dedicated hosting facility without the users noticing the move and zero downtime. Once you are on your own hardware when you manually add new nodes to your system the EIF director will immediately start making use of them.

I see this as a boon to potential startups as the will be able to grow and move resources around in real-time without needing to incur major expenses up front. Once they are making money the can build their own infrastructure at their own pace and potentially still use the AWS when the need more capacity to their own infrastructure.

Conclusion

While 2007 was a great start to the EIF and all of the applications, 2008 looks to be even better. I’m sure there will be more shifts in direction and changes made to the ideas behind the system as the start to get implemented, but the core goal of making the most flexible, stable and powerful Internet development environment will always be there.

This version is ready for testing for adventurous users that would like to help us making ejabberd the best release ever.

A detailled changelog is also available on ProcessOne bug tracker.

ejabberd 2.0.0 beta is available from ejabberd download page.

A big thank you to everyone who helped making this version what it is
currently. This is a list extracted from ejabberd's ChangeLog since it
was started 5 years ago until 2.0.0-beta1:

AV, Andreas from Unstable.nl, Andrey Zamaraev, Andrzej Smyk, Andy
Turner, Anton Vanin, Asdx, Badlop, Balabanov Dmitry, Brian Campbell,
Carlos E. Lopez, Christophe Romain, Christophe Romain, Cord Beermann,
Doruk Fisek, EQHO Communications, Etan Reisner, Evgeniy Khramtsov,
Felipe Brito Vasconcellos, Geoff Cant, Iceburn, Igor Goryachev, Jacek
Konieczny, Jean-Sebastien Pedron, Jerome Sautret, Juraj Michalek,
Justin Karneges, Karl-Johan Karlsson, Konstantin Khomoutov, Leif
Johansson, Luca Brivio, Lucius Curado, Lukas Polivka alias Spike411,
Magnus Henoch, Marina Hahn, Marshall T. Rose, Martijn van Beers,
Martin Pokorny, Marvin Preuss, Massimiliano Mirra, Max Loparyev, Maxim
Ryazanov, Michael Scherer, Michael Shields, Mickael Remond, Miguel Gomard, Milos
Svasek, Nikolaus Polak, Oleg Palij, Oleg V. Motienko, Pablo Saratxaga,
Patrick Dreker, Pedro Melo, Pieter Rautenbach, Qu1j0t3, Renato Botelho, Robin Redeker, Ruslan
Rakhmanin, Samuel Tardieu, Sander Devrieze, Sergei Golovan, Shelley
Shyan, SkLUG, Stefan de Konink, Tomas Janousek, Tomasz Sterna, Tony
Finch, Torsten Werner, Tsukasa Hamano, Usercard, Vicent Alberola,
Victor Hugo dos Santos, Vincent Ricard, Vladimir Kulev, and Zbyszek
Zolkiewski.

4 Vote(s) ]]> http://www.planeterlang.org/out.php?title=What_You_Need_To_Know_About_Amazon_SimpleDB-1 http://www.planeterlang.org/story.php?title=What_You_Need_To_Know_About_Amazon_SimpleDB-1 Sun, 16 Dec 2007 16:36:52 CET erlang http://www.planeterlang.org/story.php?title=What_You_Need_To_Know_About_Amazon_SimpleDB-1 SimpleDB is an exciting new Amazon project build in Erlang.

CHANGELOG:

• Supported OTP/Erlang R12B new syntax.


• A new Emacs standard color theme.


• Fixed some formatter bugs.


• Better syntax error message.


• Various bugs fixes.

To switch color theme, open [Tools]->[Options], click on 'Fonts & Colors', choose 'Profile' drop-down box.

Java JRE 5.0+ is required.

To download, please go to:
http://sourceforge.net/project/showfiles.php?group_id=192439

To install:

1. Unzip erlybird-bin-0.15.2-ide.zip to somewhere.


2. Make sure 'erl.exe' or 'erl' is under your environment path


3. For Windows user, execute 'bin/erlybird.exe'. For *nix user, 'bin/erlybird'.


4. Check/set your OTP path. From [Tools]->[Options], click on 'Erlang', then 'Erlang Installation' tab, fill in the full path of your 'erl.exe' or 'erl' file. For instance: "C:/erl/bin/erl.exe"


5. The default -Xmx option for jvm is set to 256M, ErlyBird now works good with less memory, such as -Xmx128M. If you want to increase/decrease it, please open the config file that is located at etc/erlybird.conf, set -J-Xmx of 'default_options'.

When run ErlyBird first time, the OTP libs will be indexed. The indexing time varies from 10 to 30 minutes deponding on your computer.

Notice:
If you have previous version ErlyBird installed, it's recommended to delete the old cache files which are located at:

• *nix: "${HOME}/.erlybird/dev"


• mac os x: "${HOME}/Library/Application Support/erlybird/dev"


• windows: "C:Documents and Settingsyourusername.erlybirddev" or some where

The status of ErlyBird is still Alpha, feedbacks and bug reports are welcome.

3 Vote(s) ]]>