Every Mac comes with an (undocumented) family member classifier

Anatomy of Apple's Personal Knowledge Graph Every Mac since macOS 13 (2022) ships a pair of daemons whose only purpose in life is to build a graph about who you are, who you know, how those people are related to you and to each other, how you communicate with them, where you go, what you do and when. In this post, we look into how this system works and give you a tool to peek into your own graph. The position A family fork: one knight on e2 attacks White's king, queen, rook and bishop at once. White's pieces are the household the graph is about; the knight is the store, one move from every one of them. White's own last move, 18.e3, emptied the square it stands on. White is in check and has one legal reply, Kh1; then the knight takes the queen. The check falls on the one who accepted the terms. What is lost is someone else. 3r1rk1/pp2qppp/8/3p1b2/8/1NQ1P1B1/PP2nPPP/2R2RK1 w - - 1 19 Finding What if I told you that every Mac since 2022 ships a pair of daemons whose only purpose in life is to build a graph about you and the people around you: who you are, who you know, how those people are related to you and to each other, how you communicate with them, where you go, with whom, what you do and when. This graph is built on device by cross-referencing pretty much everything on the machine: Contacts, Calendar, Photos, Messages, Mail, location, Screen Time and a long tail of behavioral signals. And, more importantly, it is held as a queryable store that LLMs retrieve from (allegedly). Would you believe me? Well, that's what is happening, and interestingly enough, there is little documentation of how this whole mechanism works on your Mac. The closest thing to an official description is the man page (man 8 knowledgeconstructiond), and this is all you get: NAME knowledgeconstructiond – A daemon that analyzes content on the device to build a general purpose knowledge graph. DESCRIPTION knowledgeconstructiond analyzes the content on this device to help build a knowledge graph representing the device and the user. It works together with intelligenceplatformd, performing some of the heavy processing tasks, to generate the knowledge graph. Previous work: To be fair, Apple's own researchers did publish a paper on the techniques behind this graph construction, back in 2023. But there is next to nothing on how it is integrated into macOS: which versions run it, and the exact list of what is collected and inferred. And not a word about the "kinship" inference discussed below. Also, we personally ran into it while doing some security research on the Apple Intelligence bundle in macOS 26 in August 2026; but a few months before, Nic Lydon came across the same database and wrote it up in I Built a Personal Knowledge Graph. Apple Had Already Built One on My Laptop. And even more recently, a forensics tool went looking on the phone: in September 2026, Alexis Brignoni taught iLEAPP to read the same store on iOS, where graph.db and the ontology.db that decodes its predicate codes sit under /private/var/mobile/Library/IntelligencePlatform; he writes it up in Reading the iPhone Knowledge Graph, and Checking Its Work. But we go a bit deeper. This blog post is an account of some analysis work we carried out on what the knowledge graph actually does, so that you (the user) can get a better understanding of how this mechanism works, and some clarity on what the graph holds about you and your family and friends. Everything in it was collected on macOS 26 (26.6.2 at the time of writing). To this end, we open-source a local dashboard (iKnow) you can use to read the content of your personal knowledge graph on your Mac. Finally, we derail a bit and look into what we do best: weaponizing and exploiting. We run a thought experiment (one that applies to any such personalized system) and show what happens if such a knowledge base ends up in the wrong hands, and how a "privacy single point of failure" can be exploited to threaten you in ways you may not have considered before (when combined with an abliterated LLM). What's in the personal knowledge graph? The two daemons keep their work in ~/Library/IntelligencePlatform, a directory of SQLite databases (fifty-one databases) stored in plaintext but located in a zone protected by TCC Full Disk Access. Likely, the most interesting is graph.db. It is a triple store written in a vocabulary of 181 entity classes and 278 predicates: every row is a subject, a predicate and an object. The subject is an entity (a person, an app, an event); the object is either a plain value (a name, a handle, a date) or another entity, which is where the graph gets its edges. The predicate is the interesting part: it says what kind of relationship the system believes holds between the two. One edge of the graph (synthetic data): a person and the organization the store holds as their employer. PS500 is the id predicate.json gives employer, and its range is not an organization but a relationship type — so one edge is two rows sharing a relationshipId, one naming the kind of relationship and one pointing at the other entity. The forty-six relationship type classes in the table below exist to carry exactly that, which is what lets an edge hold a confidence of its own and be retracted later. Most of what the two daemons do is populate those predicates, and the tables kept beside them, from everything the OS will let them read. Into the graph go Contacts; Photos (faces); Calendar (titles and attendees); Family Sharing (including the age of every member); the Medical ID emergency contact in Health; events and contacts Siri Suggestions finds in Mail and Messages; people donated by apps through App Intents; and installed apps. Beside it, in tables of their own, sit who appears with whom in your photos; the communication log (who, when, which app, which direction; no content), which also pulls from your other devices; location and movement; and a long tail of device signals (screen locked, Wi-Fi, Bluetooth, charging, ambient light, and a stream named ContextualUnderstanding.PhysicalSocial whose record format lists people by name, phone and email, each with a start and an end time; it reads like a log of who was physically around you, though we never saw it filled in). What is inferred The complete list of the 181 entity classes: Every class the shipped ontology declares, grouped rather than left in file order: the core types, then people, attributes, places and events, then the activity block, and last the forty-six relationship types, which are not things at all — each one reifies an edge so that the edge can carry facets of its own, a confidence among them. The ids and labels are Apple's; the grouping is ours. intValue runs 1 to 185 with four numbers missing — 18, 54, 169, 181 — and nothing on the disk says what they were. Fifty-seven of the 181 describe an activity: sleeping, exercising, mindfulness, going to bed, commute, dining, most of them with a matching type class beside them, so nearly a third of the vocabulary went on what you are doing rather than on who or where you are. That said, it is not clear that every class is actually used; some may be placeholders that are declared and never filled. In the table, we flag as implemented the classes for which we could locate a mechanism that might be responsible for the class, but those are to be taken as inference work rather than ground truth. And this is the complete list of predicates (same disclaimer applies): Every predicate the shipped ontology declares. The first column is the id as graph.db stores it; the last says how a value gets there: copied from a source, joined from another record, derived by a rule, shipped by Apple as world knowledge, or produced by a model (in bold). Seven we could not establish, and the table says so rather than guessing. The ids and labels are Apple's; the plain-language readings are ours. How it is inferred The methods run the whole range. Most of the vocabulary is filled by plain copying (a name from Contacts, an attendee from Calendar) and rules over the copies; the interesting rows come from models: a pair of XGBoost classifiers decides who is family from 144 mostly behavioral features, two embedding models are one of ten signals the system uses to decide when two records are the same person, twelve analyzers in Photos score partner, parent, child, coworker and friend from who appears in your pictures with you, on which trips and holidays, and Apple's on-device LLM reads your messages to put a name and a relationship on a face (if Apple Intelligence is active). Whatever the method, every row lands in the graph with a confidence attached, a number between 0 and 1. The family scores live apart. Two CoreML models ship next to the ontology, EntityTagging_Family.mlmodelc and EntityTagging_FamilyAndFriends.mlmodelc: binary XGBoost classifiers that take 144 features and return a class and a probability over it. Their scores do not go into the triple store but into a table of their own (entityTagging.db, one row per person, seventeen relationship columns from family to mother to coworker, recomputed every day), which is what the iKnow scorecard further down reads. Only two of those seventeen columns have a shipped model behind them; what fills the other fifteen, and how any of these probabilities would become a confidence in the graph, we could not trace. The feature names are in each model's metadata.json, in plain text. They are the clearest statement of what this system is that I have found anywhere. All 144: Every input EntityTagging_Family.mlmodelc reads, in the order its metadata.json declares them. EntityTagging_FamilyAndFriends reads the same 144 in the same order, and every one is a number. The groups are the schema's own three blocks: twenty on who the person is, forty on when and how you communicate with them, eighty-four on the photographs you both appear in. Two of the twenty, photosPersonAge and photosAgeDifferenceCategory, are read from Photos too. The names are Apple's; the group labels are ours. The identity group is the tidy story: same surname, same home address, an iCloud Family flag — and it is the small half. The bulk of the model is the other two: who you message and who appears in your photographs, conditioned on the day of the week and the time of day, and then on both at once. photosPersonPosteriorProbabilityGivenSaturdayNight is a shipped feature name in a shipped classifier whose job is deciding whether someone is family. Deleting something does not delete it everywhere. The live graph forgets. Delete a contact and nothing re-derives them; the next run replaces the table wholesale. But the store keeps a second table, expired_stable_graph, beside the live one. The retracted row moves there and keeps its confidence and its sources. How long it stays there is not clear from what we could see. It seems to go after about 48 hours (the store keeps a prune watermark, and a recent snapshot of the table reached back no further than that); but on my own Mac, I could still read very old records in it. And a conclusion can outlive the evidence for it. On my Mac, the scorecard still scored two people the graph had dropped the day before, both of them known to it only from a calendar invitation. Well, if you want your partner not to discover some contacts you might have deleted and their "closeness" to you, better to drop some rows from expired_stable_graph too (unfortunately, you have to do it manually). But I am sure you have nothing to hide (just joking). Look into your own data Is any of this sensitive? We'll leave that to you, since "sensitive" means different things to different people (though I think you have plenty to choose from here). However, the thing is, none of it lives in isolation. It is built to be joined and correlated; the nodes are there to be linked: How to read this Every node is a row in the store, and the record under it is what the store holds. Size is how many rows name it; shape is what kind of thing it is; the one node in accent is you, and a ring marks the four the store scores as family. A dashed line is a conclusion nobody typed. Click a node — or reach the graph with Tab, walk it with the arrow keys and press Enter — and its record replaces these notes. (Me) reachable at · 5 calendar record Mara Quill reachable at · 3 calendar record Tomas Quill reachable at · 2 calendar record Nils Quill reachable at · 3 calendar record Ines Quill reachable at · 2 calendar record Priya Sandoval reachable at · 3 calendar record Jonah Ferreira reachable at · 3 calendar record Lena Marchetti reachable at · 3 calendar record Sam Okafor reachable at · 2 calendar record Théo Lindqvist reachable at · 2 calendar record Greta Eriksen reachable at · 2 calendar record Iris Castellano reachable at · 2 calendar record Kai Nakamura reachable at · 3 calendar record Ona Salazar reachable at · 3 calendar record Bo Ferreira reachable at · 3 calendar record Ruth Adeyemi reachable at · 2 calendar record Ruth Tanaka reachable at · 1 Tomas Halloran reachable at · 2 Jonah Varga reachable at · 1 calendar record Nadia Brennan reachable at · 1 calendar record Oskar Lindqvist reachable at · 1 calendar record Halima Ceric reachable at · 1 calendar record Peter Vance reachable at · 1 calendar record Maya Smith reachable at · 1 calendar record Milo Fenwick reachable at · 1 calendar record Agnes Okonjo reachable at · 1 Dev Raman reachable at · 2 calendar record Clara Hovland reachable at · 2 calendar record Tobias Reiss reachable at · 1 calendar record Selma Duarte reachable at · 1 calendar record Arto Kekkonen reachable at · 1 calendar record Freya Ballard reachable at · 1 calendar record Nikhil Sundaram reachable at · 2 calendar record Marta Bielik reachable at · 2 Owen Trevelyan reachable at · 1 calendar record Ida Sorensen reachable at · 2 calendar record Rafael Pinto reachable at · 2 Noor Haddad reachable at · 1 calendar record Callum Ferris reachable at · 1 calendar record Ingrid Saetre reachable at · 1 calendar record Hugo Marchetti reachable at · 2 calendar record Alma Vicente reachable at · 2 calendar record Ben Osei reachable at · 1 calendar record Sofia Almeida reachable at · 1 calendar record Lars Ekdahl reachable at · 2 calendar record Mina Farouk reachable at · 2 calendar record Jasper Whitlow reachable at · 1 calendar record Elena Costa reachable at · 2 calendar record Andrei Pavel reachable at · 2 calendar record Tess Rowntree reachable at · 2 Karim Belkacem reachable at · 1 calendar record Verity Shaw reachable at · 2 calendar record Nils Ahlberg reachable at · 2 calendar record Leo Smith reachable at · 2 calendar record Otto Freeman reachable at · 1 calendar record Amara Diallo reachable at · 2 calendar record Gil Rosenthal reachable at · 2 calendar record Fenna de Vries reachable at · 1 calendar record Rory Macallan reachable at · 1 calendar record Sanne Bakker reachable at · 1 calendar record Luca Ferretti reachable at · 2 calendar record Esme Carrington reachable at · 1 calendar record Yusuf Demir reachable at · 2 calendar record Britt Halvorsen reachable at · 1 calendar record Nora Vasquez reachable at · 1 calendar record Emil Krahn reachable at · 2 calendar record Dahlia Mensah reachable at · 2 calendar record Ward Pelletier reachable at · 2 calendar record Sunniva Aas reachable at · 1 calendar record Joon Park reachable at · 2 calendar record Cleo Maddox reachable at · 2 calendar record Hana Petrova reachable at · 2 calendar record Bram Jansen reachable at · 1 calendar record Tiernan Walsh reachable at · 1 calendar record Zofia Mazur reachable at · 1 calendar record (unnamed) reachable at · 1 calendar record (unnamed) reachable at · 1 calendar record (unnamed) reachable at · 1 calendar record Quill household Halden Labs Bell & Marrow Verdant Coffee Co. Northwind Clinic Quill family apps carrying this chat Halden · design apps carrying this chat Five-a-side apps carrying this chat Flat 3B apps carrying this chat Thursday quiz apps carrying this chat (a chat kept by its handles) apps carrying this chat Click a node and the record under the graph is what the store holds on it (the rows, not a summary of them). The sixty-one faint ones are the part nobody expects: people the store knows and has not placed, in it because they wrote to you once, or because they were on somebody else's invitation. The most important point, though, is that this is not only about you (the person using the Mac). The people around you are in it too: your family, your friends, "anyone" you have talked to through your device. None of them was asked, and the store links them to you and, through the events and groups they share, to each other. A relationship is intelligence one level above the rows it was inferred from. For instance, by correlating across my own store, I (well, an LLM working for me) could easily work out the following: That there is a child in the household, and roughly how old. The family-member types and a family circle give the household. The scorecard's child and son/daughter columns give the child's place in it. Beside it sits an age, copied from Family Sharing; and dates of birth in the store can carry the system's own word imputed, though what the system means by it is not clear to us. Which daycare the child goes to, and both of the numbers that reach it. Copied off contact cards. Note what is not there: the calendar has no school-run event, so the store holds the institution and its lines but not the hour the child is dropped off. That is still enough to call a parent from a number their phone already trusts. Where the family will be, on which days, between which hours. One calendar event, a place, two stored time slots and a title that names the family (so the guest list is in the string even where the attendee rows are empty). Which of the family will be there. The scorecard names the ones scored as family, a household entity holds them together, and a Family Sharing circle says which of them are in the same family group. And, in the same file, what is installed on the machine, down to the password manager. It is easy to see how this could be weaponized if this single point of failure for your privacy ends up in the wrong hands. We play that (hypothetical) scenario out in a second write-up. The good thing is that this knowledge is meant to stay local on your machine, protected by the Full Disk Access permission; although it can be consumed by other services (but it is not entirely clear to us how and what can be accessed). The reality is that the amount of information in this store is really something else: every time we dig into what the daemons infer, and how, a few new lines of investigation open up. Whatever you would rather no one knew about you or your family may well be in there somewhere, and we think you have the right to know whether it is. What you get instead: no documentation, no way to inspect the store and no general way to correct an inference that is wrong. So we give you two tools. The first is iKnow, a local dashboard for browsing your own graph without anything leaving your Mac (the next section walks through it). The second is for going deeper. The iKnow repository ships a manual for your own agent that tells it where the databases are and how to read them, so it can go looking for whatever you are after. (Please point a self-hosted model at it, not a cloud one.) iKnow We designed iKnow as a web dashboard implemented in the Python standard library; no npm and no CDN. It is meant to be run directly on your Mac, where the databases are stored. When executed, it binds 127.0.0.1, locates the databases in your file system (granted the necessary permissions), opens them read-only, collects the info on the fly and represents it on a dashboard. Nothing is written and nothing is logged; values go from the stores, over loopback, into a browser tab; it can be run air-gapped if you want. We have tested it on macOS 26.6.2 and 26.7.1, and, only partially, on 27.0.1. iKnow has a dozen pages. Start here What your Mac knows. The landing page: the graph of people around you. Below it come the faces in your photos, your family, your handles, your correspondents and where you will be next. Its graph is the same household as the one at the top of this post, this time drawn by the tool's own renderer: The first screen. The white node marked (Me) is the owner of the Mac; the rest are the people, employers and group chats the store links to it, colored by what it says each one is. Orange is family, as scored. Blue is a closeness score above the cut. The faded names around the edges are people the store knows and has not placed. Nothing here is summarized that is not in the record: every node is a row, and clicking one shows you the row (which is what the graph at the top of this post does too). The graph is the overview, and an overview is easy to shrug at. The pages behind it are not. Social graph. Who the store links to whom through shared meetings, group chats and a closeness score. Each circle opens to its members and to the evidence behind it. What it concluded Kinship scorecard. The family bindings inferred by the daemon. Its card on the landing page is cut down to the people it names: Six people, six relationships, and a subtitle saying nobody typed any of it. That holds for the scores; some of what they are computed from, you did type (a relation on a contact card, an emergency contact, Family Sharing). A mother, a father and a daughter at 0.960, which is as high as these columns go; a brother, a son and a sister at 0.500. Below that is the part I find hardest to argue with. Nine separate checks (the partner column, the composite score, the household group entity, the Family Sharing token, the iCloud Family flag, top twelve-month contact, a Photos face cluster, a Contacts record, a reachable handle), and the card counts how many of them land on the same person. Eight of nine on the mother; seven on the father, and seven on Priya Sandoval, the partner, who is not in the list above: iKnow lists the relationships that roll up into another column (mother and father into parent), and partner rolls up into nothing. The join found the partner anyway. The card is a summary of one table, and iKnow draws the whole table too: Sorted by familyAndFriends, the broadest of the scores. The seven at the top each light up their own relationship and the column it rolls up into (mother and father into parent, sister and brother into sibling, son and daughter into child), and the partner is also a colleague. Further down are the friends and the rest of the office. Under them, almost everyone else is the same two cells: one repeated value on family and another on familyAndFriends, which is what the classifier writes when it has nothing to say. Two columns (alumni and unknown) are never used at all. The scores in these two shots are invented, and more generous than the real thing. On the Mac we took apart, only mother reached the top of the ladder; the few others it scored at all got one small vote on every family label at once, father and brother alike. What the fixture keeps is the table's grain: it only ever writes a handful of values (0.125, 0.25, 0.5 and 0.96), and most people sit on one default. The face index. A face cluster for everyone in your photos. Its co-occurrence graph does not use a single contact, message or calendar entry. It is built from your photographs alone: Two faces are linked when they appear in the same photograph, and that is the only input: 8,904 clustered faces over the 1,643 photographs that hold more than one. The labels are fragments of each cluster's local identifier, because there are no names to show. The owner is the hub. The knot on the left is the family, the group at the bottom right is work, and the two never share a frame. Each node wears its cluster's key face, read out of the photo library and blurred by iKnow's own switch for sharing the screen. Under the blur there is nobody: the fixture has no photo library, so every face is a synthetic portrait painted for it. This is the page's own honest point: the graph points at the face clusters and can name almost none of them (three of the twenty-nine, in the fixture). It does not need to. Who stands next to whom, how often, and which groups never meet is all there without a single name, and whoever knows the household can put the names back on in a second. Guessed or given. What you typed in and what the Mac worked out for itself live in the same table. A confidence score is the only thing separating them. What it recorded A year of contact. Who you talked to, when, and not a word of message text (and it does not need any to tell when you are asleep). Its card on the landing page is the one that gets the reaction, and it contains no inference at all. A year of contact, per person, split incoming and outgoing, with call counts and a last-seen date, across five messaging apps — and, exactly as the subtitle says, not one word of message text. None of this is content. It is all metadata, and it is enough. The last line is the one to read twice: the hours it can read as sleep. iKnow draws that line too, from the same log folded onto a single week: Every cell counts the messages and calls that fell in one hour of one weekday, over the year. The working week is in there, and so are the evenings and the late weekend mornings; and so is the pale band from one to seven, every night. Nobody stored a sleep schedule. The quietest six-hour stretch falls out of the timestamps for free. That is the general shape of this store: almost nothing in it had to be collected, because almost everything in it is a consequence of something that was. The calendar. The one layer that knows where you will be: the hour, the room, the attendees and the dial-in. What you withdrew. Withdrawing a fact does not remove it. The row moves to an expired table and keeps its confidence and provenance. The machinery The store. The file itself: what it models, and two real rows expanded into everything they touch. Device fingerprint. Every app installed on the Mac is written down in plain text, next to a token that is an unsalted SHA-256 of that same app identifier, so the page recomputes it in your browser and you can watch it match. Who else reads it. The graph is the first copy, not the only one. This page maps the views that subscribe to it, the databases they publish into, the tables that repeat the same facts, the streams coming in from hardware paired with the Mac, and the machinery that is set up but has not run yet. A note on macOS 27. macOS 27 was released on September 14, 2026. After the upgrade to 27.0.1, the graph withdrew most of the rows it had built from Contacts, Calendar, Siri Suggestions, Family Sharing and the app catalog, and rebuilt them over the following day. Some pages of iKnow may not work fully on macOS 27 yet. But let's go deeper: