[{"content":"A few weeks ago, I was working in the ER, and I saw a patient with abdominal pain who had been referred by a telemedicine provider. The patient was very distraught because she had been told that she could have a life-threatening infection in her abdomen, and she was tearful about it.\nUsually before seeing patients, I perform a \u0026ldquo;chart biopsy,\u0026rdquo; gathering some basic information from the chart on their medical history as represented in the electronic record. In this case, I could see a note from earlier today by the telemedicine provider. It was very well written, organized, and made a compelling case that the patient could have SBP, or spontaneous bacterial peritonitis, a feared complication of liver failure where bacteria from the gut infect ascitic fluid, or extra fluid around the bowels caused by liver dysfunction. I could see that this patient did have a history of liver disease, so I was immediately concerned, and began thinking through the next steps I would take, including performing diagnostic paracentesis, a procedure where a needle is introduced into the abdomen to sample the fluid to detect any infection.\nWhen I went to bedside and interviewed the patient, however, it sounded like her pain had been going on for many months, and she didn\u0026rsquo;t really have other characteristic symptoms of infection, like fever or malaise. On exam, she did have some tenderness, but I didn\u0026rsquo;t detect the characteristic bloating and \u0026ldquo;fluid wave\u0026rdquo; that would indicate a large volume of extra fluid in the abdomen.\nWhat\u0026rsquo;s Going On? # Confused, I returned to the EHR and explored her clinical notes more thoroughly. It appeared that she had been seen earlier that week and had a CT scan for the same pain, and the scan showed no fluid in her abdomen. While it\u0026rsquo;s certainly possible that a small amount of fluid could have developed since then, it\u0026rsquo;s unlikely that it would be enough to put her at risk of a serious infection. Moreover, her story didn\u0026rsquo;t support a diagnosis of infected ascitic fluid. The pain should be new and different, not going on for months.\nPuzzled, I went back to the note from the referring physician and realized my mistake.\nA Perfect Clinical Note # The note from the telemedicine provider looked excellent, and had led me to weigh it heavily. Clinical notes are hard to write, and even harder to make readable to other providers. A note of this quality must indicate a very competent clinician who was very concerned about this patient. Unconsciously, I was biased to trust the note not because of what it said, but how it smelled.\nThe Analogue of \u0026ldquo;Code Smell\u0026rdquo; # In programming circles, the term \u0026ldquo;code smell\u0026rdquo; captures the phenomenon where competent programmers can judge the quality of code based on subtle signs in how the code is written. For nearly any computational task, there are many ways to perform it algorithmically, and still more flexibility in how the algorithm is expressed in code, right down to how variable names are chosen and how lines of code are indented, organized, and commented on.\nIn emergency medicine we are perhaps the experts on assimilating clinical notes, having to scan through the chart on a patient we have probably never met trying to understand what is going on and where the important information is located. On this day, I realized that I had unconsciously developed a sense of \u0026ldquo;documentation smell,\u0026rdquo; where I judged the veracity of human-written content (not machine-generated labs or structured data) based on the length, complexity, vocabulary, completeness, organization, and tone of what the author actually chose to type (or in many cases, dictate). Surely this sense had also been honed by authoring thousands of clinical notes myself, and considered how they might be read or used in the future, choosing my words carefully.\nFool Me Once # Inspecting the well-organized note, I suddenly realized that it had almost certainly been written by an LLM, either on a bespoke healthcare platform from an enterprising startup, or perhaps a curious provider experimenting with ChatGPT.\nThe note was well-reasoned, but ignored obvious inconsistencies, like the patient\u0026rsquo;s clinical picture didn\u0026rsquo;t resemble SBP, and that she had no history of fluid accumulation in her abdomen that would place her at risk for this condition. She had a CT scan earlier in the week that had all but eliminated the possibility of such an infection, yet the note failed to address this key fact.\nMy suspicion is that the author of the note didn\u0026rsquo;t review the medical record in any detail, and whether the diagnosis of SBP originated from the clinician or an algorithm, they hadn\u0026rsquo;t thought critically about the differential diagnosis prior to sending the patient to the ER. But my oh my, their note looked and sounded beautiful!\nA Serious and Pervasive Problem # Clearly healthcare isn\u0026rsquo;t the only area where human-generated responses convey as much in their form as their content. From people writing their senator, to college admissions essays, even casual emails, the way we write conveys as much meaning as what we say, like body language on the page. And the formerly unfakeable traces of effort, \u0026ldquo;proof-of-work\u0026rdquo; to borrow a phrase, used to be something you could trust. We\u0026rsquo;re entering a world where written language, generated for centuries with great effort and by at least one human mind, now can\u0026rsquo;t be trusted by \u0026ldquo;smell.\u0026rdquo; Among the many disruptions ahead, this may be the most fundamental.\nHow to Respond? # At the moment, a critical reading can detect the tone, organization, and soulless perfection of an LLM-generated response. But not always, and with even a little prompting models can easily pass the \u0026ldquo;Turing test\u0026rdquo; of human authorship, even for vigilant readers who work with AI all the time.\nClearly, we need new ways to separate human-inspired work from algorithmic slop. But \u0026ldquo;slop\u0026rdquo; isn\u0026rsquo;t always, or even usually, sloppy in ways we can notice. As long as copy-paste is legal, how are we ever to know if a jumble of words originated from a human brain?\nThe long-derided in-person interview may need to make a resurgence. And in healthcare, it remains vital to go to the bedside and gather data from the source.\n","date":"Jul 30, 2025","externalUrl":null,"permalink":"/blog/2025-07-30-clinical-documentation-smell/","section":"Writing","summary":"A few weeks ago, I was working in the ER, and I saw a patient with abdominal pain who had been referred by a telemedicine provider. The patient was very distraught because she had been told that she could have a life-threatening infection in her abdomen, and she was tearful about it.\nUsually before seeing patients, I perform a “chart biopsy,” gathering some basic information from the chart on their medical history as represented in the electronic record. In this case, I could see a note from earlier today by the telemedicine provider. It was very well written, organized, and made a compelling case that the patient could have SBP, or spontaneous bacterial peritonitis, a feared complication of liver failure where bacteria from the gut infect ascitic fluid, or extra fluid around the bowels caused by liver dysfunction. I could see that this patient did have a history of liver disease, so I was immediately concerned, and began thinking through the next steps I would take, including performing diagnostic paracentesis, a procedure where a needle is introduced into the abdomen to sample the fluid to detect any infection.\n","title":"Loss of \"Smell\": An AI Side-Effect","type":"blog"},{"content":"","date":"Jul 30, 2025","externalUrl":null,"permalink":"/tags/artificial-intelligence/","section":"Tags","summary":"","title":"Artificial Intelligence","type":"tags"},{"content":"","date":"Jul 30, 2025","externalUrl":null,"permalink":"/tags/healthcare/","section":"Tags","summary":"","title":"Healthcare","type":"tags"},{"content":"","date":"Jul 30, 2025","externalUrl":null,"permalink":"/tags/large-language-models/","section":"Tags","summary":"","title":"Large Language Models","type":"tags"},{"content":"","date":"Jul 30, 2025","externalUrl":null,"permalink":"/tags/medicine/","section":"Tags","summary":"","title":"Medicine","type":"tags"},{"content":"Emergency physician in Boise, Idaho, working on clinical AI, clinical decision support, and healthcare quality systems.\n","date":"Jul 30, 2025","externalUrl":null,"permalink":"/","section":"Scott Gilmore, MD","summary":"Emergency physician in Boise, Idaho, working on clinical AI, clinical decision support, and healthcare quality systems.\n","title":"Scott Gilmore, MD","type":"page"},{"content":"","date":"Jul 30, 2025","externalUrl":null,"permalink":"/tags/","section":"Tags","summary":"","title":"Tags","type":"tags"},{"content":"Musings about healthcare, technology, and artificial intelligence in medicine.\n","date":"Jul 30, 2025","externalUrl":null,"permalink":"/blog/","section":"Writing","summary":"Musings about healthcare, technology, and artificial intelligence in medicine.\n","title":"Writing","type":"blog"},{"content":"","date":"Jul 15, 2024","externalUrl":null,"permalink":"/tags/ai-safety/","section":"Tags","summary":"","title":"AI Safety","type":"tags"},{"content":"","date":"Jul 15, 2024","externalUrl":null,"permalink":"/tags/explainability/","section":"Tags","summary":"","title":"Explainability","type":"tags"},{"content":"Discussions of explainability for large language models (LLMs) tend to focus on methods that identify what parts of a neural network encode certain concepts or words, with the hope that by illuminating the meaning of parts of the LLM \u0026ldquo;black box\u0026rdquo; we can be certain of how it is making a decision. These methods resemble those used in convolutional neural network architectures processing imaging data from, say, a CT scan, where parts of the source image important to the label are highlighted to help humans understand how the categorization was performed.\nBut when we talk about \u0026ldquo;explainable\u0026rdquo; systems, who is owed the explanation? While one approach may satisfy an AI engineer, it might not make sense to the user, particularly a clinician who is making a high stakes decision. Ideally the model outputs could be explained in a way that resembles a clinician\u0026rsquo;s own decision-making process.\nFor medical imaging, a \u0026ldquo;heat map\u0026rdquo; of significance makes sense, because the dimensionality of the data is quite high \u0026ndash; millions of billions of pixels are fed into the model, and they can be grouped according to their spatial relationships. But some types of clinical data are discrete, made of up of individual values like vital signs or laboratory values. Other clinical data, like a description of a pain symptom, are unstructured and occupy a very large semantic space.\nA Simple Example # Imagine this vignette:\nINPUT: 45 year-old auto mechanic with a history of hypothyroidism and peripheral vascular disease presents with left-sided chest pain that is sharp, non-radiating, not related to exertion... OUTPUT: This patient is at elevated risk for coronary artery disease. EXPLANATION: The following neural network activations led to this decision: \u0026#34;maleness\u0026#34; \u0026#34;middle-aged male\u0026#34; \u0026#34;chest pain\u0026#34; \u0026#34;vascular diseases\u0026#34; While this explanation tells us something about the clinical example, it\u0026rsquo;s quite vague, and does little to illuminate the decision-making process. What was the significance of these activations? How were they combined to arrive at the conclusion? We can see that the model assumed the auto mechanic is male, but little else of value.\nWhile it\u0026rsquo;s possible in principle to create a system that monitors the neural network activations and can provide detailed explanations of what is happening, we\u0026rsquo;re not there yet. In the meantime, is there a simpler method that would satisfy our curiosity?\nModeling Counterfactuals # Whether they admit it or not, most clinicians make decisions based on explicit or implicit algorithms based on clinical factors that are present in a case. A patient may have risk factor or clinical features that increase or decrease the suspicion for a disease. If you ask a doctor, \u0026ldquo;why did you order that test?\u0026rdquo; she can usually rattle of a few clinical features that raised her suspicion.\nIf you wanted to understand the decision more, what could you ask her? One simple approach would be to explore counterfactuals \u0026ndash; \u0026ldquo;if the patient weren\u0026rsquo;t male, would you still order the test?\u0026rdquo; But understanding what inputs would ultimately change the output, you can infer the importance of each clinical detail.\nAssuming nnn binary input data elements (yes/no data points like \u0026ldquo;male/female\u0026rdquo;) and a binary output, you could re-run the model nnn times with counterfactuals to determine the sensitivity of the output to each of these dimensions. This would require repeating the analysis nnn times, or O(n)O(n)O(n). You could explore counterfactuals involving two differences from reality with n(n−1)2\\frac{n(n-1)}{2}2n(n−1)​ iterations or O(n2)O(n^2)O(n2) time, and in general the number of iterations is just the binomial:\n(nk)=n!k!(n−k)!∈O(nk) \\binom{n}{k} = \\frac{n!}{k!(n-k)!} \\in O(n^k) (kn​)=k!(n−k)!n!​∈O(nk)Here kkk is the number of simultaneous differences from reality you wish to explore. Now O(nk)O(n^k)O(nk) isn\u0026rsquo;t impressive efficiency, but remember that the dimensionality of relevant clinical data is quite small, and counterfactuals up to order k=3k=3k=3 are probably sufficient to satisfy users.\nWe can further reduce the number of steps by asking the LLM itself, in a separate prompt and query, which clinical data are most important. The counterfactual explainer then would only need to consider the top clinical factors, so the effective nnn to consider could be much smaller.\nFor n=5n=5n=5 clinical factors deemed important, with k=3k=3k=3, we calculate the number of repeated inferences as:\n(53)=5!3!(5−3)!=10 \\binom{5}{3} = \\frac{5!}{3!(5-3)!} = 10 (35​)=3!(5−3)!5!​=10So in this reasonable clinical scenario, increasing the number of inferences by an order of magnitude is sufficient to explore the relevant counterfactuals. While this increase in compute is not a negligible cost, it seems quite reasonable if a high-stakes clinical decision is being considered.\nThe alternative, counterfactual explanation to a clinician would look like this:\nINPUT: 45 year-old auto mechanic with a history of hypothyroidism, stroke, and peripheral vascular disease presents with left-sided chest pain that is sharp, non-radiating, not related to exertion... OUTPUT: This patient is at elevated risk for coronary artery disease. CALCULATED list of clinical factors that would reverse the decision: [ peripheral vascular disease AND stroke ] EXPLANATION: If a history of peripheral vascular disease were not present AND a history of stroke were not present, this patient would not be considered high risk for coronary artery disease. As a clinician, I find this explanation much more understandable and relatable, and it would increase my comfort level in trusting the model\u0026rsquo;s outputs to know how it would respond in alternative clinical situations.\n","date":"Jul 15, 2024","externalUrl":null,"permalink":"/blog/2024-07-15-explainability/","section":"Writing","summary":"Discussions of explainability for large language models (LLMs) tend to focus on methods that identify what parts of a neural network encode certain concepts or words, with the hope that by illuminating the meaning of parts of the LLM “black box” we can be certain of how it is making a decision. These methods resemble those used in convolutional neural network architectures processing imaging data from, say, a CT scan, where parts of the source image important to the label are highlighted to help humans understand how the categorization was performed.\n","title":"Explainable Explainability for Clinical LLMs","type":"blog"},{"content":"","date":"Jul 15, 2024","externalUrl":null,"permalink":"/tags/health-care/","section":"Tags","summary":"","title":"Health Care","type":"tags"},{"content":"When we talk about implementing AI in healthcare, a lot of people envision using computers to offload part of the cognitive work related to direct patient care. Countless startups promise to reduce cost by automating, say, writing doctors notes in a more efficient way, or automatic physician messages to patients.\nBut a very large proportion of the cognitive work in health systems is not performed by providers, but the vast administrative staff who support them. But since clinicians are paid more for their time, that must be the highest value target for automation, right?\nA First-Principles Approach # Assume that there are iii roles, and the work to be performed requires nin_ini​ individuals, the initial required headcount for the iiith role. Assuming no change in the required work output, and assuming AI can perform α∈[0,1]\\alpha \\in [0,1]α∈[0,1] proportion of the iiith role, then after AI implementation:\nn~i=ni(1−αi) \\begin{equation} \\tilde{n}_i = n_i(1-\\alpha_i) \\end{equation} n~i​=ni​(1−αi​)​​Where n~i\\tilde{n}_in~i​ is the modified headcount to maintain constant work output. Then the cost savings of this change is:\nT=∑i=0mSi(ni−n~i)−niαiκ \\begin{equation} T = \\sum_{i=0}^{m} S_i(n_i - \\tilde{n}_i) - n_i\\alpha_i\\kappa \\end{equation} T=i=0∑m​Si​(ni​−n~i​)−ni​αi​κ​​Where each SiS_iSi​ is the salary of the iiith role, and κ\\kappaκ is a proportionality constant that assumes the cost of compute is proportional to the cognitive work performed. Substituting (1) into (2), we have:\nT=∑i=0mniαi(Si−κ) T = \\sum_{i=0}^{m} n_i \\alpha_i(S_i - \\kappa) T=i=0∑m​ni​αi​(Si​−κ)This captures the idea that the cost savings is proportional to the initial labor cost multiplied by the difference between the cost of human labor and the cost of automation. In the limit of negligible compute cost, we have:\nlim⁡κ→0T=∑i=0mSiniαi \\begin{equation} \\lim_{\\kappa \\rightarrow 0} T = \\sum_{i=0}^{m} S_i n_i \\alpha_i \\end{equation} κ→0lim​T=i=0∑m​Si​ni​αi​​​This suggests that in order to minimize the labor cost, one must consider the number of individuals in a role, the salary of that role, and the automation potential of that role in equal measure.\nLow-Hanging Fruit # Let\u0026rsquo;s work through a hypothetical example. Let\u0026rsquo;s say that an organization pays physicians, on average, about four times as much as administrative staff, but four administrators are needed for every physician. Let\u0026rsquo;s propose that the workload for a physician can be reduced by 10% with automation, whereas the administrator can enjoy a 50% reduction in the labor required. Which is the lower hanging fruit? Here we can write the proportion:\nSpnpαp:Sanaαa {S_p n_p \\alpha_p}:{S_a n_a \\alpha_a} Sp​np​αp​:Sa​na​αa​Where the subscripts ppp and aaa refer to physician and administrator. Plugging in the numbers from the example, we have:\n4×1×0.1:1×4×0.5 { 4 \\times 1 \\times 0.1 }:{1 \\times 4 \\times 0.5 } 4×1×0.1:1×4×0.51:5 1 : 5 1:5So with these assumptions, it would be five times as economically valuable to automate the administrator role.\nIn real life, things are much more complex, but this reality check and (3) have made me more interested in learning more about all the roles that exist in a healthcare organization, their cost, and their automation potential, because you need to understand all three to decide where to focus attention.\n","date":"Jul 7, 2024","externalUrl":null,"permalink":"/blog/2024-07-11-jobs/","section":"Writing","summary":"When we talk about implementing AI in healthcare, a lot of people envision using computers to offload part of the cognitive work related to direct patient care. Countless startups promise to reduce cost by automating, say, writing doctors notes in a more efficient way, or automatic physician messages to patients.\nBut a very large proportion of the cognitive work in health systems is not performed by providers, but the vast administrative staff who support them. But since clinicians are paid more for their time, that must be the highest value target for automation, right?\n","title":"Optimal Deployment of Automation in Healthcare","type":"blog"},{"content":"","date":"Jun 27, 2024","externalUrl":null,"permalink":"/tags/cognition/","section":"Tags","summary":"","title":"Cognition","type":"tags"},{"content":"","date":"Jun 27, 2024","externalUrl":null,"permalink":"/tags/dementia/","section":"Tags","summary":"","title":"Dementia","type":"tags"},{"content":"","date":"Jun 27, 2024","externalUrl":null,"permalink":"/tags/drug-discovery/","section":"Tags","summary":"","title":"Drug Discovery","type":"tags"},{"content":"I came across a recent podcast by Peter Attia M.D. where he interviews Dena Dubal M.D. Ph.D. on a potential \u0026ldquo;breakthrough in Alzheimer\u0026rsquo;s disease\u0026rdquo; in the form of α-klotho supplementation. α-klotho, often referred to as just \u0026ldquo;klotho,\u0026rdquo; is a transmembrane protein expressed in the brain, liver, and kidney. Klotho seems to have several mechanisms of control over different metabolic pathways, including insulin sensitivity, maintaining calcium homeostasis, mitigating oxidative stress and endothelial dysfunction, facilitating oligodendrocyte maturation, myelin integrity, and a host of other functions. 1\nKlotho was originally discovered by accident in 1997, when a researcher noticed accelerated aging in mice with a mutated gene for the klotho protein. 2 (The name \u0026ldquo;klotho\u0026rdquo; is in reference to the Greek mythological figure Clotho, who \u0026ldquo;spins the thread of human life.\u0026rdquo; 3) Since then, observational studies have shown klotho expression to somehow protect against aging and cognitive decline.\nInterestingly, klotho has been shown to be closely associated with physical exercise, such that it has been referred to as an \u0026ldquo;exerkine,\u0026rdquo; or a signaling molecule released in response to exercise. 4 There is also a human genetic variant of the klotho gene called \u0026ldquo;KL-VS\u0026rdquo; that is associated with \u0026ldquo;enhanced cognition in heterozygous carriers,\u0026rdquo; 5 further establishing klotho\u0026rsquo;s role in aging.\nDr. Dubal has been involved in research on the effects of klotho for many years, but only recently published a study of the effects of klotho supplementation in rhesus monkeys in Nature Aging (2023) 6. In the podcast, she reports the results of that study, and how it has led to the formation of Jocasta Neuroscience, Inc., a drug discovery startup that aims to explore klotho supplementation as a treatment for Alzheimer\u0026rsquo;s dementia and other forms of age-related neurocognitive disease.\nTo Good to Be True? # Here are some of the takeaways from the paper:\nLow dose (10 µg/kg) of rhesus klotho (each species has its own variant) significantly boosted cognitive function in aged rhesus macaques (see chart for high memory load task in figure labeled c below).\nHigher doses (20 and 30 µg/kg) didn\u0026rsquo;t show enhancement versus vehicle (placebo), suggesting that the effect has a relatively narrow window of efficacy with supra-therapeutic doses showing no benefit (see figures labeled g and h below).\nA single administration of 10 µg/kg enhanced performance in both normal memory load (NML) and high memory load (HML) tasks, and the effect lasted for at least two weeks (see charts for HML persistence in figure labeled d below).\nCastner et al. (2023) Castner et al. (2023) This combination of cognitive effect size and persistence of effect is very exciting because it suggests a potential biologic therapy for humans that could meaningfully improve cognition but wouldn\u0026rsquo;t require daily injections.\nExercise in a Bottle? # Given that klotho is known to be expressed by exercise, and given the strong meta-analysis support for the beneficial effect of exercise on Alzheimer\u0026rsquo;s dementia 7, could klotho be the \u0026ldquo;exerkine\u0026rdquo; mediating the beneficial effect of exercise?\nThe benefits of exercise on human health have been known for years. Exercise is sometimes not used by patients presumably because exercise can be uncomfortable and requires discipline and consistency to implement. Is it \u0026ldquo;too good to be true,\u0026rdquo; or simply \u0026ldquo;true,\u0026rdquo; that a drug therapy could deliver all or most of the benefits of exercise?\nI think it\u0026rsquo;s unlikely that we can hack our biology so easily with a single protein such that the myriad benefits of exercise are completely reproduced. But even if a fraction of the benefit can be delivered to patients who are unable or unwilling to start an exercise regimen, the beneficial effect could be massive.\nI think this an important space to watch in understanding ways to mitigate aging and meaningfully extend human longevity.\nWikipedia contributors. (2024, February 13). Klotho (biology). Wikipedia. https://en.wikipedia.org/wiki/Klotho_(biology)\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nKuro-o M, Matsumura Y, Aizawa H, Kawaguchi H, Suga T, Utsugi T, Ohyama Y, Kurabayashi M, Kaname T, Kume E, Iwasaki H, Iida A, Shiraki-Iida T, Nishikawa S, Nagai R, Nabeshima YI. Mutation of the mouse klotho gene leads to a syndrome resembling ageing. Nature. 1997 Nov 6;390(6655):45-51. doi: 10.1038/36285. PMID: 9363890.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nWikipedia contributors. (2024, February 22). Clotho. Wikipedia. https://en.wikipedia.org/wiki/Clotho\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nDe Luca Corrêa, H., Raab, A. T. O., Araújo, T. M., Deus, L. A., Reis, A. L., Honorato, F. S., Rodrigues-Silva, P. L., Neves, R. V. P., Brunetta, H. S., Da Silva Mori, M. A., Franco, O. L., \u0026amp; Rosa, T. D. S. (2022). A systematic review and meta-analysis demonstrating Klotho as an emerging exerkine. Scientific Reports, 12(1). https://doi.org/10.1038/s41598-022-22123-1\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nDubal DB, Yokoyama JS, Zhu L, Broestl L, Worden K, Wang D, Sturm VE, Kim D, Klein E, Yu GQ, Ho K, Eilertson KE, Yu L, Kuro-o M, De Jager PL, Coppola G, Small GW, Bennett DA, Kramer JH, Abraham CR, Miller BL, Mucke L. Life extension factor klotho enhances cognition. Cell Rep. 2014 May 22;7(4):1065-76. doi: 10.1016/j.celrep.2014.03.076. Epub 2014 May 10. PMID: 24813892; PMCID: PMC4176932. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4176932/\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nCastner, S. A., Gupta, S., Wang, D., Moreno, A. J., Park, C., Chen, C., Poon, Y., Groen, A., Greenberg, K., David, N., Boone, T., Baxter, M. G., Williams, G. V., \u0026amp; Dubal, D. B. (2023). Longevity factor klotho enhances cognition in aged nonhuman primates. Nature Aging, 3(8), 931–937. https://doi.org/10.1038/s43587-023-00441-x\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nLiu, W., Zhang, J., Wang, Y., Li, J., Chang, J., \u0026amp; Jia, Q. (2022). Effect of Physical exercise on Cognitive Function of Alzheimer’s Disease Patients: A Systematic Review and Meta-Analysis of Randomized Controlled Trial. Frontiers in Psychiatry, 13. https://doi.org/10.3389/fpsyt.2022.927128\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"Jun 27, 2024","externalUrl":null,"permalink":"/blog/2024-06-27-klotho-supplementation/","section":"Writing","summary":"I came across a recent podcast by Peter Attia M.D. where he interviews Dena Dubal M.D. Ph.D. on a potential “breakthrough in Alzheimer’s disease” in the form of α-klotho supplementation. α-klotho, often referred to as just “klotho,” is a transmembrane protein expressed in the brain, liver, and kidney. Klotho seems to have several mechanisms of control over different metabolic pathways, including insulin sensitivity, maintaining calcium homeostasis, mitigating oxidative stress and endothelial dysfunction, facilitating oligodendrocyte maturation, myelin integrity, and a host of other functions. 1\n","title":"Exercise in a Bottle? The Promise of α-Klotho Biologics for Age-Related Cognitive Decline","type":"blog"},{"content":"","date":"Jun 27, 2024","externalUrl":null,"permalink":"/tags/kl-vs/","section":"Tags","summary":"","title":"KL-VS","type":"tags"},{"content":"","date":"Jun 27, 2024","externalUrl":null,"permalink":"/tags/klotho/","section":"Tags","summary":"","title":"Klotho","type":"tags"},{"content":"","date":"Jun 27, 2024","externalUrl":null,"permalink":"/tags/longevity/","section":"Tags","summary":"","title":"Longevity","type":"tags"},{"content":"","date":"Jun 27, 2024","externalUrl":null,"permalink":"/tags/nootropics/","section":"Tags","summary":"","title":"Nootropics","type":"tags"},{"content":"","date":"Jun 27, 2024","externalUrl":null,"permalink":"/tags/startups/","section":"Tags","summary":"","title":"Startups","type":"tags"},{"content":"","date":"Jun 25, 2024","externalUrl":null,"permalink":"/tags/hallucinations/","section":"Tags","summary":"","title":"Hallucinations","type":"tags"},{"content":" LLMs Can Make Stuff Up # Most people who play around with ChatGPT or another LLM (and are not famous) have had the experience of asking about themselves and receiving convincing but wrong BS. Here\u0026rsquo;s what happens when I ask Llama 3 about myself:\n\u0026gt;\u0026gt;\u0026gt; Tell me about Scott Gilmore, M.D. who trained at Christiana. Scott Gilmore, MD, is an American orthopedic surgeon and researcher. He received his medical degree from Jefferson Medical College and completed his residency in orthopedic surgery at ChristianaCare Health System (then known as Christiana Healthcare System) in Wilmington, Delaware. I\u0026rsquo;m not an orthopedic surgeon (I trained in EM), and I didn\u0026rsquo;t go to Jefferson (Wake Forest), etc. etc. \u0026hellip;\nWhat\u0026rsquo;s happening here?\nBecause the LLM is trained to predict the next token, it provides the most likely details where it hasn\u0026rsquo;t encoded knowledge from data it has seen before. The LLM is trained on petabytes of published material. Most of that material is written by humans who have something to say and know what they are talking about. So the LLM mimics its training data and \u0026ldquo;tries\u0026rdquo; to show that it has something to say and that it knows what it is talking about. Even if it doesn\u0026rsquo;t.\nNot \u0026ldquo;Hallucinating,\u0026rdquo; but \u0026ldquo;Confabulating\u0026rdquo; # Colloquially, these errors are called \u0026ldquo;hallucinations,\u0026rdquo; but a recent paper by Farquhar et al. published in Nature 1 prefers the more precise term \u0026ldquo;confabulation.\u0026rdquo;\nI learned the word \u0026ldquo;confabulation\u0026rdquo; in medical school, where it was used to describe psychiatric patients who, when asked about things they couldn\u0026rsquo;t remember but should remember, made up arbitrary details, while being quite convinced of the truth of those details. The term \u0026ldquo;hallucinations,\u0026rdquo; also borrowed from psychiatry, connotes seeing fantastical, impossible things, which is kind of the opposite of perceiving plausible but inaccurate details. So \u0026ldquo;confabulation\u0026rdquo; seems to be a better term to describe what is happening here.\nConfabulation is obviously bad in two ways: not only are confabulations often arbitrary and wrong, but by definition they also sound plausible. An ideal AI chatbot would know when it was out of its depth and didn\u0026rsquo;t know something, and would restrain itself rather than duping the user with convincing falsehoods.\nHow to Detect Confabulation? # One obvious way to detect and warn the user about confabulations is to simply monitor the LLM output and sound an alarm whenever the LLM says something that is untrue. But there is an equally obvious problem: you would need to be able to fact-check all the LLM output in real-time against another source of truth. And how do we fact-check that system?\nFarquhar et al. propose a method to monitor the LLM output for confabulation based on the concept of \u0026ldquo;semantic entropy.\u0026rdquo;\nHere, \u0026ldquo;semantic\u0026rdquo; refers to the meaning of language, rather than the symbols (words, etc.) used to carry that meaning. This is a hugely important distinction because LLMs are trained to predict tokens, or specific chunks of language, rather than ideas or concepts. If an LLM correctly conveys a specific concept, it is because the specific string of words describing that concept were statistically likely to follow the model input.\n\u0026ldquo;Entropy\u0026rdquo; is a somewhat intimidating word used in a variety of disciplines. It originated in thermodynamics, where it describes the number of states a system can occupy, which is related to the concept of how \u0026ldquo;random\u0026rdquo; a system has become. Entropy was adapted for use in information theory as \u0026ldquo;Shannon entropy:\u0026rdquo; 2\nH(X)=−∑i=1nP(xi)log⁡2P(xi) H(X) = -\\sum_{i=1}^n P(x_i) \\log_2 P(x_i) H(X)=−i=1∑n​P(xi​)log2​P(xi​)Where H(X)H(X)H(X) is the entropy of a random system XXX with nnn possible states, and P(xi)P(x_i)P(xi​) is the probability of the iii-th possible state.\nConsider a penny on a table that is heads up. What is the likelihood of heads? Well, it\u0026rsquo;s 100%. So the Shannon entropy is:\nH(X)=−log⁡21=0 H(X) = -\\log_2 1 = 0 H(X)=−log2​1=0Now consider the same penny in the air, spinning as it flips in the air. If it\u0026rsquo;s a fair coin, we expect \u0026ldquo;heads\u0026rdquo; and \u0026ldquo;tails\u0026rdquo; to be equally likely:\nH(X)=−12log⁡212−12log⁡212=1 H(X) = -\\tfrac{1}{2} \\log_2 \\tfrac{1}{2} - \\tfrac{1}{2} \\log_2 \\tfrac{1}{2} = 1 H(X)=−21​log2​21​−21​log2​21​=1So there is more entropy in the system of the coin in mid-air. This definition naturally generalizes to more complex systems with more possible states.\nWhen an LLM answers a question, it chooses each token one at a time. It assigns probabilities to a set of possible tokens, and (depending on the \u0026ldquo;temperature\u0026rdquo;) chooses the most likely one. If you can see \u0026ldquo;under-the-hood\u0026rdquo; of the LLM and can know the probabilities that the model assigns to each of the candidate tokens, you can calculate the entropy at each step. For example, if you ask:\nollama run llama3 \u0026gt;\u0026gt;\u0026gt; I flip a coin. Reply \u0026#34;heads\u0026#34; or \u0026#34;tails.\u0026#34; The \u0026ldquo;under-the-hood\u0026rdquo; next token probabilities might look like this:\nTOKEN PROBABILITY heads 0.49 tails 0.48 i can\u0026#39;t do that dave 0.01 (This is a little imprecise, because token\u0026rsquo;s aren\u0026rsquo;t words or phrases, but that distinction isn\u0026rsquo;t important here.)\nIn the example of the known coin value, things would look like this:\nollama run llama3 \u0026gt;\u0026gt;\u0026gt; A coin is heads up. Is it heads or tails? Reply \u0026#34;heads\u0026#34; or \u0026#34;tails.\u0026#34; TOKEN PROBABILITY heads 0.98 tails 0.01 what? 0.01 In this second case, there is much less \u0026ldquo;randomness\u0026rdquo; \u0026ndash; the LLM will almost always answer \u0026ldquo;heads.\u0026rdquo; If you calculate the entropy, it will be close to zero. And why? Because \u0026ldquo;heads\u0026rdquo; is the right answer.\nPut another way, if you\u0026rsquo;re making up arbitrary details, there are a lot of possibilities to choose from. But if you actually know what is true, and are trying your best to report it, you will be much more constrained in what you can say, and so your reply will be less random.\nBack to \u0026ldquo;Semantics\u0026rdquo; # So what\u0026rsquo;s with semantics? Can\u0026rsquo;t we just use the entropy calculations above to decide if the LLM knows what it\u0026rsquo;s talking about?\nThe reason is because in human language, there are many ways to say the same thing. In fact, the probable responses to the above question might have been:\nTOKEN PROBABILITY heads 0.70 it\u0026#39;s heads 0.20 the answer is heads 0.15 obviously heads 0.05 If we calculate the entropy naively using these probabilities, it will look like the answer is quite random because model could choose to say \u0026ldquo;heads\u0026rdquo; in many different ways. But as humans reading the response, we can see that all these answers are the same. More precisely, they have the same meaning, or are \u0026ldquo;semantically equivalent.\u0026rdquo;\nNoticing this, Farquhar et al. have proposed a solution: create a system that looks at the LLM responses and counts them as identical if they have the same meaning. Then you can calculate the entropy based on the possible meanings or concepts. Then the entropy of the response should truly correlate with whether the response is \u0026ldquo;true,\u0026rdquo; or constrained by the facts, rather than \u0026ldquo;confabulated,\u0026rdquo; or unconstrained by the facts (i.e. \u0026ldquo;BS\u0026rdquo;).\nIn their paper, they claim that their methods based on semantic entropy are more accuate than previously described methods at identifying when an LLM doesn\u0026rsquo;t really know what it\u0026rsquo;s talking about.\nIs This Practical? # But wouldn\u0026rsquo;t this system of checking LLM responses add massive additional layers of computation, rendering responses frustratingly slow?\nIn short, no. The first step, calculating the probabilities of the sequences of tokens, is already performed by the LLM in its normal operation. That layer just needs to be exposed to the fact-checking algorithm.\nThe second step, clustering the responses according to their meaning, can be done efficiently because it\u0026rsquo;s comparably easy to determine if two statements are \u0026ldquo;entail\u0026rdquo; each other (i.e. if XXX is true, YYY is also true, and vise versa, which is a good measure of semantic equivalence.) In the paper, the authors used several methods, including one based on GPT 3.5, that performed similarly humans at clustering answers according to semantic equivalence.\nIs This Interesting? # Besides offering a clear and intuitive system for judging the reliability of LLM responses, this hints at the next frontier of AI models.\nA common objection when anyone tries to say LLMs constitute artificial general intelligence is that these models are \u0026ldquo;just predicting the next token\u0026rdquo; and not doing any real thinking.\nBut if an AI system is trying to predict the next concept\u0026hellip; well, that sounds a lot like how I wrote this article with my brain. Besides just resembling human thought, systems that manipulate ideas and concepts rather than chunks of text can potentially be faster, more compressible, and more reliable. Ultimately what we care about is the meaning of language, and advancements that hinge on the distinction between symbols and and their meaning hints at bringing us closer to systems that truly emulate human cognition.\nFarquhar, S., Kossen, J., Kuhn, L. et al. Detecting hallucinations in large language models using semantic entropy. Nature 630, 625–630 (2024). https://doi.org/10.1038/s41586-024-07421-0\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nWikipedia contributors. (2024, June 7). Entropy (information theory). Wikipedia. https://en.wikipedia.org/wiki/Entropy_(information_theory)\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"Jun 25, 2024","externalUrl":null,"permalink":"/blog/2024-06-25-semantic-entropy/","section":"Writing","summary":"LLMs Can Make Stuff Up # Most people who play around with ChatGPT or another LLM (and are not famous) have had the experience of asking about themselves and receiving convincing but wrong BS. Here’s what happens when I ask Llama 3 about myself:\n\u003e\u003e\u003e Tell me about Scott Gilmore, M.D. who trained at Christiana. Scott Gilmore, MD, is an American orthopedic surgeon and researcher. He received his medical degree from Jefferson Medical College and completed his residency in orthopedic surgery at ChristianaCare Health System (then known as Christiana Healthcare System) in Wilmington, Delaware. I’m not an orthopedic surgeon (I trained in EM), and I didn’t go to Jefferson (Wake Forest), etc. etc. …\n","title":"Semantic Entropy to Detect Confabulation in Large Language Models (Farquhar et al.)","type":"blog"},{"content":"","date":"Jun 25, 2024","externalUrl":null,"permalink":"/tags/statistics/","section":"Tags","summary":"","title":"Statistics","type":"tags"},{"content":"","externalUrl":null,"permalink":"/categories/","section":"Categories","summary":"","title":"Categories","type":"categories"},{"content":" Leave this empty: Name * Email * Subject * Message * 0 of 1000 max characters Submit ","externalUrl":null,"permalink":"/contact/","section":"Scott Gilmore, MD","summary":" Leave this empty: Name * Email * Subject * Message * 0 of 1000 max characters Submit ","title":"Contact","type":"page"},{"content":" Startups # Impathiq, Inc.\nGliant, L.L.C.\nClinical Training # Christiana Care\nEmergency Medicine Residency\nNewark, Delaware\nEducation # Wake Forest School of Medicine\nDoctorate of Medicine\nWinston-Salem, North Carolina\nPrinceton University\nAB in Physics\nPrinceton, New Jersey\nPublications # “Fluctuation of Public Interest in COVID-19 in the United States: Retrospective Analysis of Google Trends Search Data.” Husain I, Briggs B, Lefebvre C, Cline DM, Stopyra JP, O’Brien MC, Vaithi R, Gilmore S, Countryman C. JMIR Public Health Surveill 2020;6(3):e19969.\n“Can Machine Learning Predict Outcomes in Patients with Infection?” Gilmore, Scott M. MD, Arnold, Ryan MD, Capan, Muge PhD Poster Presentation Christiana Care Research Symposium September 2018\n","externalUrl":null,"permalink":"/cv/","section":"Scott Gilmore, MD","summary":"Startups # Impathiq, Inc.\nGliant, L.L.C.\nClinical Training # Christiana Care\nEmergency Medicine Residency\nNewark, Delaware\nEducation # Wake Forest School of Medicine\nDoctorate of Medicine\nWinston-Salem, North Carolina\n","title":"Curriculum Vitae","type":"page"},{"content":"Clinical-AI demonstrations, technical experiments, open-source work, and architecture papers.\n","externalUrl":null,"permalink":"/projects/","section":"Projects","summary":"Clinical-AI demonstrations, technical experiments, open-source work, and architecture papers.\n","title":"Projects","type":"projects"},{"content":" Publications # “Fluctuation of Public Interest in COVID-19 in the United States: Retrospective Analysis of Google Trends Search Data” Husain I, Briggs B, Lefebvre C, Cline DM, Stopyra JP, O\u0026rsquo;Brien MC, Vaithi R, Gilmore S, Countryman C. JMIR Public Health Surveill 2020;6(3):e19969.\n“Can Machine Learning Predict Outcomes in Patients with Infection?” Gilmore, Scott M. MD, Arnold, Ryan MD, Capan, Muge PhD Poster Presentation Christiana Care Research Symposium September 2018\n","externalUrl":null,"permalink":"/research/","section":"Scott Gilmore, MD","summary":"Publications # “Fluctuation of Public Interest in COVID-19 in the United States: Retrospective Analysis of Google Trends Search Data” Husain I, Briggs B, Lefebvre C, Cline DM, Stopyra JP, O’Brien MC, Vaithi R, Gilmore S, Countryman C. JMIR Public Health Surveill 2020;6(3):e19969.\n“Can Machine Learning Predict Outcomes in Patients with Infection?” Gilmore, Scott M. MD, Arnold, Ryan MD, Capan, Muge PhD Poster Presentation Christiana Care Research Symposium September 2018\n","title":"Research","type":"page"},{"content":"","externalUrl":null,"permalink":"/series/","section":"Series","summary":"","title":"Series","type":"series"},{"content":" The Euler-Lagrange Equation # Here is a derivation of the Euler-Lagrange equation using the calculus of variations. We begin with an optimal path between aaa and bbb, with an added perturbation as follows:\nx(t)=x0(t)+ϵ⋅η(t) x(t) = {x_0}(t) + \\epsilon \\cdot \\eta(t) x(t)=x0​(t)+ϵ⋅η(t)x˙(t)=x0˙(t)+ϵ⋅η˙(t) \\dot{x}(t) = \\dot{x_0}(t) + \\epsilon \\cdot \\dot{\\eta}(t) x˙(t)=x0​˙​(t)+ϵ⋅η˙​(t)We are interested in the path that optimizes a functional, so:\n0=∂S[x(t),x˙(t)]∂ϵ∣ϵ=0 0 = \\frac{\\partial S[x(t), \\dot{x}(t)]}{\\partial \\epsilon} \\big|_{\\epsilon = 0} 0=∂ϵ∂S[x(t),x˙(t)]​​ϵ=0​=∫ab∂L[x(t),x˙(t)]∂ϵdt∣ϵ=0 = \\int_a^b \\frac{\\partial L[x(t), \\dot{x}(t)]}{\\partial \\epsilon} dt \\big|_{\\epsilon = 0} =∫ab​∂ϵ∂L[x(t),x˙(t)]​dt​ϵ=0​=∫ab∂L∂x(t)∂x(t)∂ϵ+∂L∂x˙(t)∂x˙(t)∂ϵdt∣ϵ=0 = \\int_a^b \\frac{\\partial L }{\\partial x(t)} \\frac{\\partial x(t)}{\\partial \\epsilon} + \\frac{\\partial L }{\\partial \\dot{x}(t)} \\frac{\\partial\\dot{x}(t)}{\\partial \\epsilon} dt \\big|_{\\epsilon=0}=∫ab​∂x(t)∂L​∂ϵ∂x(t)​+∂x˙(t)∂L​∂ϵ∂x˙(t)​dt​ϵ=0​=∫ab∂L∂x(t)η(t)+∂L∂x˙(t)η˙(t)dt = \\int_a^b \\frac{\\partial L }{\\partial x(t)} \\eta(t) + \\frac{\\partial L }{\\partial \\dot{x}(t)} \\dot{\\eta}(t) dt =∫ab​∂x(t)∂L​η(t)+∂x˙(t)∂L​η˙​(t)dtIntegrating by parts, we have:\n=∫ab∂L∂x(t)η(t)dt+∂L∂x˙(t)η(t)∣ab−∫abddt∂L∂x˙(t)dt = \\int_a^b \\frac{\\partial L }{\\partial x(t)} \\eta(t) dt + \\frac{\\partial L}{\\partial \\dot{x}(t)}\\eta(t)\\big|_{a}^{b} - \\int_a^b \\frac{d}{dt}\\frac{\\partial L}{\\partial \\dot{x}(t)} dt =∫ab​∂x(t)∂L​η(t)dt+∂x˙(t)∂L​η(t)​ab​−∫ab​dtd​∂x˙(t)∂L​dtWe know that η(t)\\eta(t)η(t) is zero at the limits of integration aaa and bbb, so:\n0=∫abη(t)[∂L∂x(t)−ddt∂L∂x˙(t)]dt 0 = \\int_a^b \\eta(t) [\\frac{\\partial L }{\\partial x(t)} - \\frac{d}{dt} \\frac{\\partial L}{\\partial \\dot{x}(t)}] dt 0=∫ab​η(t)[∂x(t)∂L​−dtd​∂x˙(t)∂L​]dtSince η(t)\\eta(t)η(t) is an arbitrary function, for the integral to be always zero, the rest of the integrand must always be zero, thus:\n∂L∂x(t)−ddt∂L∂x˙(t) \\frac{\\partial L }{\\partial x(t)} - \\frac{d}{dt} \\frac{\\partial L}{\\partial \\dot{x}(t)} ∂x(t)∂L​−dtd​∂x˙(t)∂L​which is the Euler-Lagrange equation.\n","externalUrl":null,"permalink":"/teaching/","section":"Scott Gilmore, MD","summary":"The Euler-Lagrange Equation # Here is a derivation of the Euler-Lagrange equation using the calculus of variations. We begin with an optimal path between aaa and bbb, with an added perturbation as follows:\nx(t)=x0(t)+ϵ⋅η(t) x(t) = {x_0}(t) + \\epsilon \\cdot \\eta(t) x(t)=x0​(t)+ϵ⋅η(t)x˙(t)=x0˙(t)+ϵ⋅η˙(t) \\dot{x}(t) = \\dot{x_0}(t) + \\epsilon \\cdot \\dot{\\eta}(t) x˙(t)=x0​˙​(t)+ϵ⋅η˙​(t)We are interested in the path that optimizes a functional, so:\n","title":"Teaching","type":"page"}]