I know it's a joke, but I really wish it was true

( You are about to view content that may not be appropriate for minors. )

I've just been catching up on the last six months of business news with respect to the pending Google Anti-Trust lawsuit. I've lived through two rounds of the US Government taking on technology companies, IBM in the 80s and MS in the 90s.

IBM, as a result of the anti-trust threat being held over it's head became incredibly afraid of innovation, so much so that they are a consulting house these days. MS lost their dictator in chief, and have drifted listlessly ever since.

Unlike both IBM and MS, Google is not a technology company. Both IBM and MS made their money in their respective heydays by selling IT solutions. In IBM's case, it was turnkey hardware, in MS's, software suites. Google, for it's thirty thousand brilliant software engineers, doesn't make money selling IT solutions.

Google is a rent extraction company, living large on the firehose of money that goes by the rubric of "Advertising Spend." Advertising is huge, in the US it hovers around 2% of GDP, and has done for a century, give or take.

I suspect the current round of increased visibility of Google's internal processes is an indirect result of this Anti-Trust threat. We are seeing more google open source software, we are seeing more talks about how google builds it's data centers and more interaction with the linux kernel community. The flow of knowledge out of google's tech centers will have a massive impact on the future of technology use in industry.

So while an anti-trust shakedown of Google is imminent, and will do untold damage to Google the company, like it did to MS and IBM before them, I think the increase in knowledge in the widespread community will be a godsend.

Spider Man understands the internet

Cecily Janus hosts a glorious writers retreat in America and is bringing it to Australia!

Others coming with her are Gary Heidt from Signature Literary Agent, Justin Taylor from HarperCollins, Kevin Doughtten from Penguin, Michael Signorelli from HarperCollins, Timothy O'Connell from Random House and Robert Guinsler from Sterling Lord Literistic Agency.

So what is it? A 15,000 word manuscript consult, one on one with a publisher or agent. 

Extract from PDF:
Traditional conferences offer you minutes, if you’re lucky, with professionals you need to connect with. If you get those precious minutes with them, often times, they have no idea what you’re doing as far as your writing nor do they know your goals as a writer. As a writer at Writing Away, you’ll receive feedback on over 15,000 words of your manuscript from each of the talents.

By the time you arrive, your manuscript has already been read, analyzed and feedback is completed. And this feedback isn’t given in the form of a letter...It’s given in the form of a one-on-one consultation that has no set limits on time nor is limited to cramming in too many authors in one day.

To further the experience, all of the talent stays in the same location with you. They eat meals with you, take time to get to know you and figure out what would be best for you as the next step in your career is launched.

**SPACE IS LIMITED! SIGN UP NOW. $1,000 USD down is required to hold your place after acceptance through registration process and payment plan available but all fees must be paid within 90 days of reservation. I am willing to work with writers on payments so that if need is demonstrated, they may attend. EARLY BIRD DISCOUNT: 5% off of total cost if registered by Jan, 15th 2010.

Next Saturday at 11am on A Writer Chat Rooms, we’re having the host, the publisher from Penguin and possibly one of the agents to come on and chat not just about the retreat, but the industry. Chat is free, of course. You don’t even have to consider the workshop to come along to it.

The chat on Saturday now also includes Michael Signorelli from Harper Collins and agent Robert Guinsler

To get to the chat, go to http://www.awritergoesonajourney.com/, hover over 'WriteSpace' and click on Chat Rooms. You do NOT need to sign up on site to join in on the chat (and ignore the bit about asking for a password.)

OR

http://awritergoesonajourney.com/chat/FlashChat_v4712/chat/flashchat.php

Notes from me: There's heaps more information on the webpage located here:

http://www.awritergoesonajourney.com/index.php?option=com_myblog&show=blog-title-click-to-edit.html&Itemid=254

Gosh, one more week! Still finalising things, but it's all happening. If any one would like to be my support person for Saturday, please feel free to let me know.
:-)

I have just posted an amended version of last year's What To Bring List. Let me know if I have missed anything!

( You are about to view content that may not be appropriate for minors. )

One thing I learnt from this video of exploding capacitors was that all those dodgy scenes in which complex racks of equipment explode as soon as something goes wrong aren't necessarily impossible, just really implausible. (video from Make via boingboing, who point out that the fun bit starts at 2:30)

( You are about to view content that may not be appropriate for minors. )

I'm going to understand this code base if it's the last thing I do.

ps: I didn't make it out for halloween. So no pics of me tradgoth'd. Does that mean I'm banned off my own journal? =)

http://www.aaronsw.com/weblog/adlogic

When should you buy ads? Let’s assume your goal is for people to click on the ads and give you money. (Reasons this may not be true: persuasion, brand-building, budget-maximizing.) The return from a block of ads is thus revenue - marginal_costs - ad_costs. Ads are an investment like any other; you keep buying them until your return on investment (revenue - marginal_costs / ad_costs) equals your cost of capital (usually the interest rate).

For simplicity, we’ll assume your marginal cost is zero. (My marginal cost is almost always zero, so this doesn’t strike me as too unrealistic.) So how do you estimate revenue? You can track how much money people who click on your ad give you, but this has two flaws. First, customers often give you more money over time. Maybe they buy level one of your video game when they click on the ad, but then they may buy levels two and three the next day after they beat level one. The future is always in the future, so revenue-per-user numbers may be too small.

Second, they might have given you money anyway. Your video game ads probably run on video game review sites, where readers might buy your game just from the review, even if you hadn’t bought an ad. So your revenue numbers may be too big.

But these problems aren’t so serious. In the first case, the worst that happens is you don’t buy as many ads as you should. In the second, you don’t actually lose money, it’s just that some extra profit you could have kept has gone into ads.

Let’s turn to the ad seller. They probably want to maximize how much they charge per ad impression (CPM). (Reasons this may not be true: unseemly ads.) A good way to do this is to hold an auction. It’s impractical to have everyone bid live, so Google auctions work like eBay auctions: you enter the maximum you’re willing to pay and get charged just enough to beat the other bidders. (One can think of this as a computer-simulated auction where everyone keeps bidding up the price by pennies until they hit the maximum they’re willing to spend.)

But what are you bidding on? Ad sellers want to maximize revenue per impression, but ad buyers want to maximize profit per expense. In an ideal world, ad sellers auction off impressions (this is what Google Ad Manager does) while ad buyers bid per dollar of profit (entering their cost of capital).

Determining how much profit you make from an ad is hard. Can we just trust you? Let’s say you make $2 in profit per 1000 impressions and everyone else makes $1. Now you can lie and say you make $1 in profit and then pay twice as much per profit-dollar. Now you pay the same amount as before, but you win all the profit-dollar auctions. Now that’s not wrong — you’re clearly making more money than the other bidders, so you should win — but your bid isn’t cost-per-profit anymore, it’s cost-per-impression.

What if you paid based on revenue? Verifying revenue is difficult, but Google could do it if everyone was using Google Checkout. (If you sent some of your users to a non-Google Checkout system, Google could catch you and fine you.) Google offers nicer ads to Checkout users, but they still don’t have much market share, making this system impractical at present.

Some search engines apparently had cost-per-action (CPA) auctions, where you paid based on how many people actually bought things. I have no idea how they made that work, since lying about how many people took an action seems really profitable and easy. Maybe that’s why no one does this anymore.

That just leaves cost-per-click (CPC). Cost-per-click seems ideal, since it’s verifiable by both the ad seller (who uses a redirect link to track clicks) and the ad buyer (who sees the users show up on their page). It’s a nice half-way point between buyer and seller.

So the ad seller holds an auction for CPC and multiplies CPC by click-thru-rate (CTR) to calculate CPM. They shows the highest CPM ads, charging each the bidder below them’s CPC, times their relative CTRs. (In reality, Google doesn’t just use CTR; they also factor in the relevance of the ad and the quality of the page it goes to.) And, voila: we’ve derived the basics of an online ad system.

This works out great for the ad seller — they maximize CPM, just like they wanted — but the ad buyer is still stuck converting their ROI into CPC. The ad buyer, recall, wants to increase their spending on ads (now determined to be CPC) until their return on investment equals their cost of capital.

It seems like this should be pretty easy, and indeed Google does provide tools to calculate ROI, but apparently not to optimize it. What they do provide is a tool to optimize your cost-per-action. Does anyone know why this is?

It seems like an automatic ROI optimizer would lead many people to spend more money on ads. It’s hard to believe Google is leaving all that money on the table.

But Google does intelligently optimize the ads themselves. The variance in click-thru rates between different ads is huge — it’s not uncommon to see two very similar ads, but one gets ten times as many clicks as the others. Google lets you put in as many ads as you like and automatically rotates them, showing ads with better CTRs more often.

So far we’ve just had a single ad seller. In the real world, lots of people want to sell ads and lots of people want to buy them. How do you match them up?

One option is make the buyer choose. This is how Google Search works: Google holds an auction for each search query and buyers pick which ones they want to compete in. Another is to group related websites together and run ads evenly across all of them. This is how most smaller ad networks work. And then there’s AdSense. AdSense scans a page for relevant keywords, then runs the Google Search ads that won auctions for those keywords.

Google also knows a lot about ad viewers. By tracking what web pages you visit, they know what topics you’re interested in. I’m apparently interested in Unix, the environment, elections, government, and social science, so Google prefers to show ads on those subjects to me.

But there’s another way to think about ad matching: as a giant optimization problem. Which combinations of user, ad placement, and advertisement optimize click-thru rates (or, ultimately, ROI)?

For each of these, there are lots of variables. For each user, you know their history, geographical location, computer (browser, operating system, screen size), ISP, etc. For each ad placement, you know time of day, hosting website, page content, etc. And for each ad, there are numerous possible variations in phrasing and design that can be tested, as mentioned before.

The possible combinations are infinite. You can’t test all of them, so you need to come up with ones that are plausible. You can look at which combinations worked in the past: has this ad done significantly better in some cities than others, or at some times than others? And you can look for patterns across ads: do ads that do well on CNN also do well on MSNBC? These hypotheses can then be tested and, if they work, you start running ads more there.

Netflix claims they’ve made millions from slight improvements in their movie recommendations.¹ When they offered a prize for more, researchers found thousands of tiny patterns and came up with all sorts of innovative algorithms to try to get an edge. After 32 months, researchers doubled the algorithm’s effectiveness.

Imagine how much more is at stake for Google. Last year, they received $21 billion in ad revenue, of which 60% was apparently profit. Even tiny improvements would be worth the highest salaries — a 0.004% improvement would make $500,000. Doubling it would create unspeakable wealth.

Yet Google has no contest for improving ad click-thru rates. Indeed, press reports suggest they don’t even have an internal team working on it. The AdWords user-interface (recently redesigned from jaw-droppingly wretched to just wretched) would seem to suggest they don’t do this kind of optimization at all. Their blog asks people to optimize things manually. No doubt there are some things humans (even ad purchase reps) can do better than computers, but surely there’s a lot more they can do together — with humans giving the machine additional hints and hypotheses to test. But there doesn’t seem to be anything like that.

It’s hard to believe this is true. It’s hard to believe this can last.

Google’s chief economist claims that Google’s sewn up the ad market by being better than everyone else. What if you made an ad network that was better than Google?

Right now Google takes a 20% cut of every auction price. What if you were willing to take just 10%? You could give ad sellers a slightly higher CPM — they’d gladly run your ads when they paid more and Google’s the rest of the time. Then you can offer ad buyers a slightly lower CPC. As long as the money people made was more than the cost of setting things up, they’d switch. I’m actually not sure why this hasn’t happened.

Now imagine that you were a genius CS student who could come up with a better ad optimization algorithm. Your system would have a higher overall CTR, since it presented users with better ads. This means that, again, you can pay higher CPMs (since more people click per impression). And you can redirect some of the money you would spend on higher CPMs into lower CPCs, to attract advertisers.

But to develop the algorithms and do the optimization you need the data. Lots of it — lots of users, lots of advertisers, lots of ad spots. No startup will ever have that; it’s only left to Google (or whichever giant eventually replaces them).

I’m not normally one to be too concerned about improving Google’s bottom line (they seem to be doing alright), but as an ad buyer I’m frustrated I have to do this work myself. I’d rather solve the problem for everyone. And if Google wants to pay me for that, I certainly wouldn’t mind.

http://www.aaronsw.com/weblog/election09

I hope that you all vote today if you can.

As the left gained power, many cities switched to off-year elections and non-partisan candidacy. Removing party affiliations from the ballot servers as a kind of poll tax, it forces people to spend time researching each candidate individually instead of just knowing they support a particular party. For those in Cambridge, MA, I have tried to help out by doing the research for you.

Cambridge has the additional complication of having a decent voting system, so you can list your candidates in order of preference. There are two questions on the ballot. For city council, there seem to be three basic categories: People with good ideas, people with no ideas, and people with bad ideas. I have listed them in that order:

1. Lawrence J. Adkins (more public services, affordable housing and health)
2. Mark Flanagan (homeless shelter)
3. Larry Ward (inclusionary zoning)
4. James M. Williamson (elected mayor, street nusiances)
5. Kenneth E. Reeves (Harlem Children’s Zone)
6. Charles Marquardt (fire the city manager)
7. Gregg Moree (energy efficiency, living wage)
8. Kathy Podgers (housing vouchers, parks)
9. Tim Toomey (fuel efficient vehicls)
10. Marjorie Decker (community engagement)
11. Neal Leavitt (achievement gap)
12. Silvia Glick (neighborhood protection)
13. Sam Seidel (afterschool)
14. Henrietta Davis (goo-goo)
15. E. Denise Simmons (311)
16. Minka vanBeuzekom (transparency)
17. Tom Stohlman (do nothing)
18. Craig Kelley (traffic enforcement, no TV for kids)
19. Leland Cheung (promote entrepreneurship, school reform)
20. David Maher (segregated schools, service cuts)
21. Edward Sullivan (tough on crime, homeland security)

School issues are inevitably depressing. Everyone says they oppose the achievement gap and so on, so my first test was to see how people felt about standardized tests (ordered from opposition to support):

1. Marc McGovern (community education centers)
2. Alice Turkel (portfolios, high quality preschool)
3. Richard Harding (no high-stakes)
4. Nancy Tauber (no teaching to the test)
5. Patty Nolan (no drill and kill)
6. Alan Steinert, Jr. (tests are “something to be endured”)
7. Joseph Grassi (desegregation)
8. Fred Fantini (more assessment)
9. Charles Stead, Sr. (tight ship principal)

Happy Election Day!

UPDATE: The winners were (in order): 9, 14, 15, 20, 5, 18, 19, 13, 21. And: 4, 3, 1, 8, 5, 2.

( You are about to view content that may not be appropriate for minors. )

It's just under 2 weeks away now! I hope you're all feeling excited! 

It's supercheap this year at $80, with day passes at $35. We'll provide tea, coffee, milks and fruit, and it's up to you to provide your own food.
:-) 

We're at Bickley Brook again this year, so we all know it well! Feel free to check out the programme

Highlights this year include discussions on Goal Planning/Life Planning, Art, Home Preserving and Holidays. Activities include skipping rope, bocce, and once again, art!
:-)

If you're coming, or thinking about coming, please come over and fill out our poll on tea, coffee, and attendance. We're hoping to have a tea sampling party one afternoon, and would really love your input! 

( You are about to view content that may not be appropriate for minors. )

( You are about to view content that may not be appropriate for minors. )

A very interesting reply in a comment thread on MetaFilter answering the question "Does math have big scary teeth or something?"

This reply draws out something very interesting. It seems the traditional working definition of "smart" is "being able to memorise lots of trivia." A more interesting definition of smart, to me at least, is "being able to work things out." Trouble shoot. Be creative. Find a way.

I suppose this is why i really dig creatives of every stripe, be they graphics geeks, font nerds, musos, photographers, etc. I respect people who can see something before it is built, and find a way to pull it into existance, kicking and screaming the whole fracking way.

Another thing starts to make sense as well. I've heard many stories where in, say, a group of doctors are having a conference to figure out what illness a patient has, and after hours of deliberation, come to a conclusion. (It's not lupus.) Then as they go about their duties, a nurse pipes up with "Was it X?" Dumbfounded, the doctors ask, how did you know? "I googled the symptoms" is the reply.

Google is destroying the value of being able to memorise facts.

( You are about to view content that may not be appropriate for minors. )

( You are about to view content that may not be appropriate for minors. )

The Answer Factory: Fast, Disposable, and Profitable as Hell: A wired article that talks about content made to order based on search data and adsense predictions. In short, cut the price on content production until it matches what you can reap from adsense. Pulp.

So this is why the internet is dying. It's becoming a wasteland of cheaply produced crap.

Mmmm.