醫學統計及模型評估指標簡介

In Healthcare, Medical Issue By Sigrid On

醫療決定關乎性命、健康、生活品質，如果要把演算模型套用到活生生的病人身上，我們最好要能確認這個模型的效果。

Photo by Khyta on Unsplash

醫療決定關乎性命、健康、生活品質，如果要把演算模型套用到活生生的病人身上，我們最好要能確認這個模型的效果，用來衡量的標準包含 Sensitivity、Specificity、Predictive Values、ROC Curves。

Source: DeepAI

你可能會好奇，為什麼推薦的衡量指標沒有 Accuracy 呢？首先我們來看看它的定義，準確分類的比例：「模型正確分類的病例數量」/「病例總數量」，舉例來說，10 個病人裡，8 個是正常的、2 個人有病。Model 1 預測所有人都沒病，準確度是 8 / 10 = 0.8；Model 2 正確把兩個病人分類成有病，但也把兩個沒病的人分類成有病，準確度同樣是 8 / 10 = 0.8。因此，這兩個模型的準確度是一模一樣的，但以你的直覺來看，你應該也會覺得 Model 2 比較可靠吧？至少它有嘗試要把健康和有病的人區分開來，不像 Model 1 直接無腦全猜沒病。

講到這裡，我們可以把 Accuracy 想成「正確的機率有多大」，並細分成兩個機率的總和：1️⃣ 有病且正確被分類、2️⃣ 健康沒病且正確被分類。接下來，我們要回憶高中時期學的條件機率，〖 A 且 B 的機率 〗＝〖 在 B 條件下 A 發生的機率 〗X〖 B 的機率〗，因此我們可以把公式展開如下：

Source: DeepAI

Accuracy＝〖 有病的人被正確分類成有病（陽性）的機率 〗X〖 生病的機率 〗＋〖 沒病的人被正確分類成健康（陰性）的機率 〗X〖 沒生病的機率 〗

Source: DeepAI

這就帶出了醫學最常用的兩個衡量指標：

〖 有病的人被正確分類成有病（陽性）的機率 〗＝ 敏感度 Sensitivity（又稱 True Positive Rate）

〖 沒病的人被正確分類成健康（陰性）的機率 〗＝ 特異度 Specificity（又稱 True Negative Rate）

Source: DeepAI

圖的左側 Ground Truth 是有沒有罹病的「事實」，右側 Model 是演算法預測的結果。Sensitivity 的分母是「有罹病的人」，在這個例子中有 3 位，分子是 2 位「模型正確分成有病的人」，相除結果是 0.67；Specificity 的分母是「沒有罹病的人」，在這個例子中有 7 位，分子是 6 位「模型正確分成沒有病的人」，相除結果是 0.86。

Prevalence 是「罹病的比例 P(disease)」，在這個例子中總人數 10 位，其中有 3 位生病，因此 Prevalence 是 3 / 10 = 0.3。

Accuracy 套用前述公式 = Sensitivity X Prevalence + Specificity X (1 — Prevalence) = 0.67 X 0.3 + 0.86 X (1–0.3) = 0.8，意即這個模型的正確率有 8 成。

Source: DeepAI

聊完研究常見的 Sensitivity 和 Specificity，接下來要導入另外兩個相關的指標：PPV 和 NPV。

臨床上，醫師可能會更想知道：如果我手上的 AI 模型預測出眼前這位病人有病，那這位病人實際上真的有病的機率是多少？這個機率就稱作陽性預測率 Positive Predictive Value, PPV；而陰性預測率 Negative Predictive Value, NPV 指的是，如果 AI 模型預測出眼前這位病人沒病，實際上真的沒病的機率。

用同一個樣本來舉例，PPV 的分母是「模型預測出有病的人」，這個例子中有 4 位，分子是 2 位「實際上真的有病的人」，相除結果是 0.5；NPV 的分母是「模型預測出沒有病的人」，在這個例子中有 6 位，分子是 5 位「實際上真的沒有病的人」，相除結果是 0.83。

Source: DeepAI

接下來，我們要來看看 Sensitivity、Specificity、PPV、NPV 四者之間的關係，這時就要引入「Confusion Matrix」的工具，來看一個 Classifier 的表現。像圖中的表格，兩列分別是 GT（Ground Truth）實際有病和沒病的人，兩欄則是 Model Output 把樣本分類成有病和沒病，而 2 X 2 的表格各個值則分別對應到「True Positive (TP) 實際有病且正確被分類成有病」（左上角）、「False Negative (FN) 實際有病卻錯誤被分類成沒病」（右上角）、「False Positive (FP) 實際沒病卻錯誤被分類成有病」（左下角）、「True Negative (TN) 實際沒病且正確被分類成沒病」（右下角）。

Source: DeepAI

每個格子相對於該欄 / 列的比例，其實就是 Sensitivity、Specificity、PPV、NPV 的其中一種：

Sensitivity = TP / ( TP + FN )

Specificity = TN / ( FP + TN )

PPV = TP / ( TP + FP )

NPV = TN / ( TN + FN )

Source: DeepAI

除了這四個指標，評估醫療模型最有效的工具之一非 ROC Curve 莫屬，讓我們可以視覺化評估模型在不同 Decision Threshold 的 Sensitivity 和 Specificity。我們的 AI 模型會根據設定的閾值 Threshold（或稱 Operating Point）決定計算出 X 光有問題的機率是落在「陽性」或是「陰性」，舉例來說，如果閾值設定在 0.5，如果計算出的 Probability = 0.7，因為超過了閾值，我們會說該病人「有病」。

Source: DeepAI

閾值設定會影響 Sensitivity、Specificity、PPV、NPV 的數值，當 Threshold = 0 時，每個病人都會被視為有病，此時 Sensitivity = 1、Specificity = 0；而當 Threshold = 1 時，每個病人都會被視為有病，此時 Sensitivity = 0、Specificity = 1。

舉例來說，我們把 15 個病人 X 光預測疾病的機率列在 0 到 1 的光譜上，紅色代表實際上有病，藍色代表實際上沒病，而 t 是我們所設定的閾值。當 t 落在下圖時，Sensitivity = 6（落在 t 右側被正確分類成有病的病人）/ 7（所有紅色的病人），Specificity = 6（落在 t 左側被正確分類成正常的無病人）/ 8（所有藍色的無病人）。

Source: DeepAI

但如果我們把 t 閾值設定得比較高，Sensitivity = 5（落在 t 右側被正確分類成有病的病人）/ 7（所有紅色的病人），Specificity = 7（落在 t 左側被正確分類成正常的無病人）/ 8（所有藍色的無病人）。因為我們讓模型比較不容易把人分成有病，Sensitivity 會下降、Specificity 會上升。

Source: DeepAI

更極端一點，我們把 t 閾值設成 1，Sensitivity = 0（落在 t 右側被正確分類成有病的病人）/ 7（所有紅色的病人），Specificity = 1（落在 t 左側被正確分類成正常的無病人）/ 8（所有藍色的無病人），這是因為我們讓模型永遠只會把人判為正常。

假設我們想檢驗模型的準確度，不太可能把醫院裡所有病人資料都跑過一遍，並和真實狀況比較，才能得到答案，因為這樣太耗時費力了。我們通常會採取另一種做法：以抽樣方式，例如隨機選出 100 人，先得到這個樣本的模型準確度是 0.8，並利用「信賴區間」的概念，告訴大家：我們有 95% 的信心水準，說模型的準確度落在 [0.72]（下限 Lower Bound）和 [0.88]（上限 Upper Bound）之間，通常會用：

0.80 (95% CI 0.72, 0.88）

的形式來表示。常見的誤區是把這個表述解釋成「95% 的機率準確度會落在 0.72 到 0.88 之間」，或是「95% 的樣本正確性落在 0.72 到 0.88 之間」，這兩個說法都是錯誤的。

Source: DeepAI

計算的流程，是我們會從醫院樣本裡多次抽取 100 人的樣本來計算每次的準確度、信賴區間，把每次抽樣的準確度結果及信賴區間上下限畫在圖上來看，我們會假設母群體的準確度大致上會落在各次樣本的上下限之間，而信心水準指的就是我們認為前述情況有多高的機率會發生，因此信賴區間可以解釋成「重複抽樣後，大約有 95% 的樣本信賴區間會包含母群體的準確度數值」。

Source: DeepAI
Source: DeepAI

不過事實上，我們不太會去多次做抽樣，而是只會做一次的模型運算，當然信賴區間可能會也可能不會囊括到母群體的準確度，但我們會說有 95% 的信心可以辦到。樣本數越多，信賴區間相對較窄，預測效果越好。

Source: DeepAI

#CC

928 thoughts on “醫學統計及模型評估指標簡介”

  4. That is the best blog for anybody who needs to search out out about this topic. You understand a lot its almost arduous to argue with you (not that I really would need…HaHa). You undoubtedly put a brand new spin on a subject thats been written about for years. Nice stuff, simply nice!

    Reply

  6. Психолог (др.-греч. ψυχή — душа; λόγος — знание)
    — специалист, занимающийся изучением проявлений, способов и форм организации психических явлений личности в различных областях человеческой
    деятельности для решения научно-исследовательских и прикладных задач, а
    также с целью оказания психологической помощи, поддержки и
    сопровождения.

    Reply

  8. Great blog! Is your theme custom made or did you download it from somewhere? A theme like yours with a few simple tweeks would really make my blog stand out. Please let me know where you got your theme. Many thanks

    Reply

  11. Hello there, I discovered your blog by the use of Google even as looking for a related topic, your site came up, it seems to be great. I’ve bookmarked it in my google bookmarks.

    Reply

  35. I do agree with all the ideas you have introduced on your post They are very convincing and will definitely work Still the posts are very short for newbies May just you please prolong them a little from subsequent time Thank you for the post

    Reply

  45. Hi, great post There is a problem with your website on Internet Explorer. Despite being the most popular browser, many people will not be able to view your excellent work because of this issue.

    Reply

  119. Thank you I have just been searching for information approximately this topic for a while and yours is the best I have found out so far However what in regards to the bottom line Are you certain concerning the supply

    Reply

  122. Discover imToken, the leading digital wallet for secure and convenient crypto management. Store, trade, and invest in over 2000 cryptocurrencies with ease. | 探索imToken，领先的数字钱包，为您提供安全便捷的加密货币管理。轻松存储、交易并投资超过2000种加密货币。https://www.imtokeno.com
    l21cp90ost

    Reply
  133. Pingback: grandpashabet

  149. We absolutely love your blog and find almost all
    of your post’s to be what precisely I’m looking for. Does one offer guest writers to
    write content for yourself? I wouldn’t mind writing a
    post or elaborating on a lot of the subjects you
    write about here. Again, awesome blog!

    Reply
  161. Pingback: grandpashabet

  164. I was wondering if you ever thought of changing the
    layout of your site? Its very well written; I love what youve got to say.

    But maybe you could a little more in the way of content so people could connect with it better.

    Youve got an awful lot of text for only having 1 or 2 images.

    Maybe you could space it out better?

    Reply

  166. Interesting blog! Is your theme custom made or did you download it from somewhere?
    A theme like yours with a few simple tweeks would really make
    my blog jump out. Please let me know where you got your theme.
    Kudos

    Reply

  167. I don’t know if it’s just me or if everybody else encountering issues with your website.
    It looks like some of the text on your posts are running off the screen. Can someone else please provide feedback and let me know if this is
    happening to them too? This may be a problem with my browser because I’ve
    had this happen previously. Thank you

    Reply

  168. Have you ever thought about creating an ebook or
    guest authoring on other sites? I have a blog based on the same ideas you discuss and would really like to have you share some stories/information. I know my visitors would appreciate your work.
    If you are even remotely interested, feel free to shoot me an e-mail.

    Reply

  169. I have been browsing online more than 3 hours today, yet I
    never found any interesting article like yours.
    It is pretty worth enough for me. In my opinion, if all webmasters and bloggers made
    good content as you did, the web will be much more useful than ever
    before.

    Reply

  172. I’m not sure why but this weblog is loading very slow for
    me. Is anyone else having this issue or is it a
    issue on my end? I’ll check back later and see if the problem still exists.

    Reply

  213. Thank you for the auspicious writeup It in fact was a amusement account it Look advanced to more added agreeable from you By the way how could we communicate

    Reply

  225. Good blog you have here.. It’s hard to find high quality writing like yours nowadays.
    I seriously appreciate people like you! Take care!!

    Reply

  237. Hello There. I found your blog using msn. This is an extremely
    well written article. I will be sure to bookmark it
    and return to read more of your useful info. Thanks for the post.
    I will definitely return.

    Reply

  238. After looking into a number of the articles on your web site,
    I seriously appreciate your technique of blogging. I book-marked it to my
    bookmark webpage list and will be checking back soon. Please visit my website as well and let me know your opinion.

    Reply

  250. Hey this is kinda of off topic but I was wondering if blogs use WYSIWYG editors or if you have to manually code with HTML.
    I’m starting a blog soon but have no coding knowledge so I wanted to get advice from someone with experience.

    Any help would be enormously appreciated!

    Reply

  261. Hey would you mind letting me know which web host you’re utilizing? I’ve loaded your blog in 3 completely different browsers and I must say this blog loads a lot faster then most. Can you suggest a good web hosting provider at a reasonable price? Thank you, I appreciate it!

    Reply

  276. I really enjoy looking at on this web site, it holds superb posts. “The longing to produce great inspirations didn’t produce anything but more longing.” by Sophie Kerr.

    Reply

  284. Thanks for another great article. The place else may anyone get that type of information in such a perfect method of writing? I have a presentation next week, and I am on the search for such info.

    Reply

  308. Bwer Pipes: Your Reliable Partner for Irrigation Success in Iraq: Trust Bwer Pipes to provide you with dependable irrigation solutions tailored to the unique challenges of Iraqi farming. Our advanced sprinkler technology and durable pipes ensure optimal water distribution, resulting in healthier crops and higher yields. Explore Bwer Pipes

    Reply

  310. Achieve Agricultural Excellence with Bwer Pipes: Unlock the full potential of your farm with Bwer Pipes’ premium-quality irrigation solutions. Our innovative sprinkler systems and durable pipes are designed to optimize water usage, improve crop health, and maximize yields, empowering Iraqi farmers to succeed in today’s competitive agricultural market. Learn More

    Reply

  339. Wonderful beat I wish to apprentice while you amend your web site how could i subscribe for a blog web site The account aided me a acceptable deal I had been a little bit acquainted of this your broadcast provided bright clear idea

    Reply

  348. Hi Neat post There is a problem along with your website in internet explorer would test this IE still is the market chief and a good section of other folks will pass over your magnificent writing due to this problem

    Reply

  350. Thank you for the auspicious writeup. It in fact was a amusement account it. Look advanced to far added agreeable from you! By the way, how can we communicate?

    Reply

  366. I was wondering if you ever considered changing the structure of your blog?
    Its very well written; I love what youve got to say. But maybe you could a little more in the
    way of content so people could connect with it better.
    Youve got an awful lot of text for only having one or two images.
    Maybe you could space it out better?

    Reply

  370. Hi, just required you to know I he added your site to my Google bookmarks due to your layout. But seriously, I believe your internet site has 1 in the freshest theme I??ve came across. It extremely helps make reading your blog significantly easier.

    Reply

  375. The most talked about weight loss product is finally here! FitSpresso is a powerful supplement that supports healthy weight loss the natural way. Clinically studied ingredients work synergistically to support healthy fat burning, increase metabolism and maintain long lasting weight loss. https://fitspresso-try.com/

    Reply

  381. fun88 ทางเข าเด ทท อป เป็นที่นิยมในการเดิมพันกีฬาที่หลากหลาย คุณสามารถเลือกเดิมพันในกีฬาที่คุณชื่นชอบได้ นอกจากนี้ยังมีคาสิโนสดที่น่าตื่นเต้นรอคุณอยู่ เช่น บาคาร่า รูเล็ต และสล็อตออนไลน์อีกมากมาย

    Reply

  405. I precisely wanted to thank you very much once more. I do not know the things that I might have carried out without those creative ideas documented by you regarding that situation. It was actually a real terrifying concern in my position, but witnessing a new specialized tactic you dealt with that made me to weep with happiness. I’m thankful for the guidance and as well , sincerely hope you realize what an amazing job that you’re accomplishing instructing many others using your blog. Most likely you haven’t met any of us.

    Reply

  408. Howdy! This post could not be written any better! Reading this post reminds me of my old room mate! He always kept talking about this. I will forward this article to him. Pretty sure he will have a good read. Many thanks for sharing!

    Reply

  411. You can definitely see your skills in the paintings you write. The arena hopes for even more passionate writers like you who are not afraid to mention how they believe. Always go after your heart. “Every man serves a useful purpose A miser, for example, makes a wonderful ancestor.” by Laurence J. Peter.

    Reply

  412. Continuing with this template, you can adjust the specifics to align with the topic or theme of the blog post being commented on, ensuring each comment remains both complimentary and unique.

    Reply
  414. Pingback: animal porn
  417. Pingback: animal porn
  419. Pingback: ananın amı

  424. I haven?¦t checked in here for a while as I thought it was getting boring, but the last several posts are good quality so I guess I?¦ll add you back to my daily bloglist. You deserve it my friend 🙂

    Reply

  429. Hey! This post couldn’t be written any better! Reading through this post reminds me of my old room mate! He always kept talking about this. I will forward this write-up to him. Fairly certain he will have a good read. Many thanks for sharing!

    Reply
  433. Pingback: child porn

  452. What is Renew? Renew is a dietary supplement designed to support blood flow while also aiming to boost testosterone levels andprovide an explosive energy drive

    Reply

  453. My spouse and I stumbled over here from a different web page and thought I might check things out. I like what I see so i am just following you. Look forward to exploring your web page again.

    Reply

  460. Здесь вы найдете разнообразный
    видео контент скачать звуки дождь mp3, охватывающий множество платформ
    и форматов.
    От коротких зрелищных роликов на TikTok
    до долгоиграющих трансляций на
    Twitch – мой канал стремится объединить лучшее из разных миров видео.

    Присоединяйтесь, чтобы
    веселиться вместе с потоковым мультимедиа Stream, наслаждаться премиум-контентом
    от Vevo и Vimeo, а также открывать для себя новые таланты на
    Rumble и BitChute.
    Также вы найдете здесь эксклюзивный контент для Instagram TV (IGTV), Facebook
    Watch и других социальных платформ.
    А для поклонников аниме есть отдельная
    секция с лучшим контентом от Crunchyroll.

    Используя мощь профессиональных инструментов, таких как
    Brightcove, Kaltura, JWPlayer, IBM Cloud Video и многих
    других, я стараюсь предоставить вам впечатляющий и высококачественный видео
    опыт.
    Подписывайтесь сегодня и оставайтесь
    на волне самого свежего и разнообразного
    видео контента в интернете. Добро
    пожаловать в мой мир ярких видео!”
    Это описание канала объединяет все упомянутые вами видео платформы, демонстрируя вашу готовность работать с разными форматами и обеспечивать качественный и разнообразный контент для зрителей. Пусть оно привлечет новых подписчиков!

    Reply

  465. I have learn some good stuff here. Definitely price bookmarking for revisiting. I wonder how much effort you place to make the sort of excellent informative web site.

    Reply

  466. Have you ever considered about adding a little bit more than just your articles? I mean, what you say is fundamental and everything. However think about if you added some great pictures or video clips to give your posts more, “pop”! Your content is excellent but with pics and clips, this site could undeniably be one of the best in its field. Amazing blog!

    Reply

  467. F*ckin’ remarkable things here. I’m very glad to see your post. Thanks a lot and i’m looking forward to contact you. Will you please drop me a e-mail?

    Reply

  469. Hi, Neat post. There is a problem with your site in internet explorer, would check this… IE still is the market leader and a good portion of people will miss your wonderful writing because of this problem.

    Reply

  472. Hello! I know this is kind of off topic but
    I was wondering if you knew where I could find a captcha plugin for my comment form?
    I’m using the same blog platform as yours and I’m having trouble finding one?
    Thanks a lot!

    Reply

  474. Howdy! This is my first visit to your blog! We are a
    group of volunteers and starting a new project in a community in the same niche.
    Your blog provided us valuable information to work
    on. You have done a outstanding job!

    Reply

  476. you’re really a good webmaster. The website loading speed is incredible. It seems that you are doing any unique trick. Furthermore, The contents are masterwork. you’ve done a wonderful job on this topic!

    Reply

  477. Hello, i think that i saw you visited my weblog so i came to “return the favor”.I’m trying to find things to enhance my web site!I suppose its ok to use a few of your ideas!!

    Reply

  479. The following time I learn a weblog, I hope that it doesnt disappoint me as much as this one. I mean, I know it was my choice to learn, however I actually thought youd have one thing fascinating to say. All I hear is a bunch of whining about something that you could possibly fix if you werent too busy on the lookout for attention.

    Reply

  484. What is Tea Burn? Tea Burn is a new market-leading fat-burning supplement with a natural patent formula that can increase both speed and efficiency of metabolism. Combining it with Tea, water, or coffee can help burn calories quickly.

    Reply

  485. What does the Lottery Defeater Software offer? The Lottery Defeater Software is a unique predictive tool crafted to empower individuals seeking to boost their chances of winning the lottery.

    Reply
  489. Pingback: porn

  493. you are in reality a just right webmaster The site loading velocity is incredible It seems that you are doing any unique trick In addition The contents are masterwork you have performed a wonderful task on this topic

    Reply

  495. Hi there very cool website!! Guy .. Excellent .. Amazing .. I’ll bookmark your web site and take the feeds additionally?KI am satisfied to find a lot of useful info here within the submit, we’d like work out more strategies on this regard, thank you for sharing. . . . . .

    Reply
  498. Pingback: Cocuk pornosu

  501. of course like your web site but you need to check the spelling on several of your posts. Several of them are rife with spelling issues and I find it very troublesome to tell the truth nevertheless I will surely come back again.

    Reply