Conversation with #gubug at 2005-01-10 12:28:57 on macnewbold@irc.freenode.net (irc)
(12:28:57) : The topic for #gubug is: roBust, Solid, Dependable
(12:55:24) pfctl!~pfctl@anthonychavez.org: pfctl has changed the topic to: roBust, Solid, Dependable -- http://www.anthonychavez.org/images/haxorpc.jpg
(13:56:22) macnewbold: pfctl: _very_ funny! rotflol!
(13:58:33) macnewbold: where'd you get that?
(14:03:12) pfctl: macnewbold: A friend of mine found it.  No idea where. ;-)
(14:03:21) macnewbold: very cool
(14:03:50) pfctl: Yeah, we laughed about it all day yesterday. hehe
(17:54:29) fungus left the room (quit: ).
(00:49:26) jlp: what a grueling day
(07:58:11) macnewbold: jlp: how did things go?
(08:36:27) jlp: I thought it went quite well... had some real mind bending questions
(08:36:37) jlp: I'll find out thursday or so
(08:36:47) jlp: could have an offer as early as next tuesday if they want to move forward
(09:25:15) fungus [~fungus@firebat.aros.net] entered the room.
(09:49:24) macnewbold: So if they offer and you accept, would you have to move to CA?
(09:50:07) pfctl: G'morning, folks.  Go, jlp!
(09:50:29) pfctl: jlp: What were some of the q's?
(09:59:57) ***pfctl watches ACM TechNews email pile up in his inbox.
(10:43:21) pfctl: macnewbold: busy?  mind if I /msg you?
(10:43:34) macnewbold: go ahead, pfctl
(17:36:28) jnbek left the room (quit: Remote closed the connection).
(17:39:38) jnbek [~jnbek@c-67-166-97-223.client.comcast.net] entered the room.
(18:26:25) fungus left the room (quit: ).
(19:03:07) jlp: mac, yes, I'd have to move to CA... at great expense
(19:03:07) jlp: pfctl, there were many questions which are all in a big fog in my head right now... :-)
(19:03:07) jlp: here's an example for you to bend your brain around
(19:03:15) jlp: let's say you have a couple of sites, site A and site B, which both have facilities to serve up www.foo.com
(19:03:49) jlp: each site has it's own AS, and it's own IP block.  using the dns, what can you do to ensure that customer experiences are good?
(19:04:24) jlp: (say, for the sake of the argument, that site A is in the US and siteB is in Asia, just so you know what "good" user experience means)
(19:47:16) jlp: then, say, for example, AS537, which has been talking to site A, suddenly can't see site A any more (some weird routing flap, whatever)... it can see B, but of course, it doesn't know it wants to talk to A
(19:47:30) jlp: how can you, at foo.com, detect that this has happened, and what can you do about it?
(08:40:02) macnewbold: jlp: just a guess, but can you run a DNS server at each site that resolves www.foo.com to it's own site's servers? Then assume that if the first DNS server is unreachable, they'll ask the second one?
(09:18:18) fungus [~fungus@firebat.aros.net] entered the room.
(09:19:13) jlp: that's part of the answer to the first problem, but the issue with that is that most resolver libraries will alternate between dns servers, picking the first one at random, so that model actually results in splitting the traffic equally between the sites (with about half the users hitting the wrong one)
(10:00:25) macnewbold: so do the dns servers give different answers depending on the src IP of the request?
(10:01:01) macnewbold: but how can you tell if the src IP can't currently reach the other site?
(10:06:41) jlp: yes, and cleverly :-)
(10:09:28) jlp: the first part of the question requires your dns servers to be "clever"
(10:10:09) jlp: if you don't know the source IP yet, you start out by serving up the A records with a very low ttl 
(10:10:38) jlp: then you do some tests... maybe ping the originating dns server, maybe run a traceroute (from both sites, of course, your dns servers have to talk to each other)
(10:11:45) jlp: gives you a good idea of which of your sites would be best for the user.  once you have a good idea (even a relatively good idea is probably good enough), you roll up the ttl on your responses, handing back the "best" A record
(10:12:37) jlp: I'll let you think about the second part of the question for a while longer :-)