[ HOW WE DO THIS ]
Short version: we type questions into ChatGPT every day and write down the answers. The longer version is below, because if you're going to trust a list, you should know exactly where it came from.
We ask real questions. Not questions we made up — questions people actually type when they're looking for an agent. "Who are the best real estate agents in Park Slope?" "Who should I hire to sell a brownstone in Brooklyn?" We used search data to find the ones people ask most, and we wrote them the way people write them.
We ask from a blank account. This part matters. If you ask ChatGPT something on your own account, the answer is shaped by your past chats, your location, and anything it remembers about you. We ask through a developer connection that starts fresh every time — no history, no memory, no profile. It's the closest you can get to a plain, unbiased answer.
We publish the answer word for word. The names, the order they came in, the firms, and ChatGPT's own one-line reason for each pick. We don't add anyone, we don't remove anyone, and we don't reorder. Every page shows the date we asked. We keep every previous day's answer too.
We ask again every day. These answers change. Someone who's first today may be gone next week. That's not a flaw in the list — that's the thing worth watching.
Right now we only ask ChatGPT. Google's Gemini and Perplexity are coming, and when they arrive each list will say which chatbot gave that answer.
Nobody can pay to be on a list. This site is run by Position Strategy Group, a company that gets paid to help businesses show up better in AI answers. That's a real conflict of interest, so here's exactly how we handle it: the lists are produced by a program that publishes automatically, with no human approving them first. Clients of that company get nothing here — no placement, no edits, no early look. If we ever run a paid feature, it will say "sponsored" in large letters. The lists never will.
Who's behind this. Check The Scoreboard was built by Josh Popkin, who runs Position Strategy Group. He's written up why he started it — what he was trying to find out, and what he expected to be wrong about — here: Who is Josh Popkin and why did he create Check The Scoreboard?
What these lists are not. They're a record of what a chatbot said on a given day. They are not our opinion of anyone, and they're not a measure of who's actually good at their job. ChatGPT gets things wrong. Sometimes it names a company when you asked for a person, or repeats something out of date. We publish that too, because how often it's wrong is part of what's worth knowing.