Fixing Multi-Sub-Domain SEO Dillusion
Some friends of mine work for a company that have an ASP model service that resides in JavaScript and HTML space on their customer’s websites. They have a problem though - because their content technically resides on another subdomain, and is just included in JavaScript via an iframe, it dilutes a lot of the link juice for their clients. Try as they might, there’s really no good way to force the two subdomains into one simply by creating lots of links to one another or any other such nonsense.
So the obvious next step is to bring the two together programmatically. We started by looking at how you could use a proxy to combine the two. You put a proxy on www.abc.com that says whenever you see something pointing to www.abc.com/xyz go pull the equivalent content from www.xyz.com. This gets trickier though, because in this case www.xyz.com is using content caching networks. Alas, another headache. Now they have to use DNS to go and find otu where www.xyz.com is - and they can’t cache that DNS request because who knows if that content caching network will go offline.
I got into some early conversations with F5 about this, and there’s a possibility that they may actually want to get into this game. So here’s why it doesn’t currently work and why it may in the future. Right now F5 uses “pools” of IP addresses to represent a single host name. In this case instead of a hostname to a pool of IP address mapping, you need a URL to a hostname mapping. Using something similar to how they do load balancing uptime detection they instead need to identify where the host www.xyz.com lives. Now you may be asking yourselves where does the uptime detection come into play? Well, thankfully in this case, it doesn’t need to be anything more than what is currently happening.
Because the F5 is making outbound requests and getting answers back it automatically knows if the site is down or slow, by virtue of the fact that it isn’t getting it’s responses back in time. So it can automatically adjust and move over to the next IP address by making another DNS request. In this way, both subdomains end up on the same domain from a user’s perspective - and more importantly, from the search engine’s perspective. If you don’t have time to wait the months or years it might take to get this built into the F5, or you can’t afford an F5, look at hacking up a CGI proxy. That’s probably your next best bet.