Redis failure handling

[fractaloop]

Users need the ability to handle scenarios where Redis goes down, causing the internal redis-client actor to die.

Recipient[Actor[akka://TestActorSystem/user/redis-client-0#-1167390260]] had already been terminated.

I've added a test that demonstrates the failure.

[aktasonur]

Creating a router with a resizer helps a bit but does not fix the problem completely, it is causing too many reconnections.

private lazy val host:String = app.configuration.getString("redis.host").getOrElse("localhost")
private lazy val port:Int = app.configuration.getInt("redis.port").getOrElse(6379)
private lazy val remote = new InetSocketAddress(host, port)

// Create min 5, max 32 clients
val clientRef = AkkaManager.system.actorOf(RedisConnection.props(remote, RedisClientSettings()).withRouter(SmallestMailboxRouter().withResizer(new DefaultResizer(5, 32))))

def client = {
    val client = new RedisClient(clientRef)
    client
}

I get following error almost every call

[INFO] [04/11/2014 11:48:26.441] [actors-akka.actor.default-dispatcher-83] [akka://actors/user/$f/$~] Child termination detected: Actor[akka://actors/user/$f/$~/pipeline#356865062]
[INFO] [04/11/2014 11:51:30.831] [actors-akka.actor.default-dispatcher-103] [akka://actors/deadLetters] Message [com.redis.protocol.ConnectionCommands$Select] from Actor[akka://actors/temp/$i] to Actor[akka://actors/deadLetters] was not delivered. [10] dead letters encountered, no more dead letters will be logged. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [04/11/2014 11:51:30.969] [actors-akka.actor.default-dispatcher-103] [akka://actors/user/$f/$wc] Connected to redis server *****
[INFO] [04/11/2014 11:51:31.108] [actors-akka.actor.default-dispatcher-106] [akka://actors/user/$f/$wc] Child termination detected: Actor[akka://actors/user/$f/$wc/pipeline#366308140]

Any suggestions?

[fractaloop]

This has been completed with #74. I'm so happy we got this feature finished!