Dealing with Leeches

One of the foundational principles of spaced repetition is separating easy cards from hard cards. We can then review the easy cards rarely and the hard cards more often. This is great.

Unfortunately, some cards are not just hard, but really hard, so hard that if we don’t do something about them, we can keep forgetting and relearning them over and over again, causing them to consume a ridiculous amount of our study time while providing little benefit. The spaced-repetition community calls these extremely difficult cards leeches because they suck away your study time.

Anki identifies a card as a leech for you when it sees that you’ve learned it and then forgotten it again a total of eight times (this threshold is adjustable). Anki will then tell you it found a leech and suspend the card so it doesn’t show up for review again until you take some action. At this point, it’s up to you to decide what action is appropriate.

In this in-depth journey into leeches (this sounds like a Magic School Bus episode!), we’ll start by discussing how to prevent leeches from arising in the first place, then we’ll see what approaches you can take to fix them when they do sneak in, and finally we’ll quickly review the mechanics of finding leeches and taking action on them in Anki.

Preventing leeches

The best way to manage leeches is to prevent them from developing in the first place. By the time you’ve forgotten a card eight times, you’ll likely have gone in circles reviewing the card several dozen times, meaning you’ve already wasted quite a bit of review time on it. Making your cards precise, following the Twenty Rules (see previous link for details), and immediately editing cards when you notice they have precision issues will prevent many prospective leeches from ever becoming a problem at all.

If you’re not ready to immediately fix a problematic card you encounter during review, mark and suspend it so it stops consuming your review time until you have time to fix it. On AnkiMobile, there’s an action called “mark and suspend”, which I’ve assigned to a downward swipe in the settings to make this faster. On the desktop version, press *@ – pronounced “splat-at,” if that helps you remember – while reviewing to mark and suspend the current card. To later find and fix marked cards, search for tag:marked in the browser.

Area for research: I suspect Anki’s leech-detection algorithm could be improved to reduce the impact of as-yet-undiscovered leeches on your study time – it might be as simple as tweaking the threshold to be lower than eight lapses. However, I’ll refrain from recommending or even suggesting anything here since I have never experimented with this myself.

Six-step decision framework

Now on to handling identified leeches. First, you should attempt to figure out why you keep forgetting the answer! Once you understand that, you can, in broad outline, reformulate the questions to make them easier, learn some background information you’re missing, delete questions that aren’t important, deprioritize questions that you don’t immediately need to know the answers to, or create mnemonics to make remembering easier.

To help with this process, here’s a set of questions to ask yourself, in order, when processing a leech. You don’t have to follow it exactly, but it can’t hurt to keep the list by your desk until you develop an intuitive understanding of how to handle leeches.

Summary

First, I’ll give a high-level overview of the questions, then we’ll dive into the details of each.

1. Is there an obvious reason I am frequently forgetting this card?
   • For instance, perhaps it’s unclear what the question is asking, or it does not follow the Minimum Information Principle.
   • If so: Edit the card and return it to review. Occasionally you may also have to add one or more new cards.
2. Can I shore up the question by adding additional cards?
   • You may be missing important context, or experiencing memory interference, which you can clear up by learning the answers to one or more pointed questions.
   • If so: Add one or more cards for additional context, then return the original card to review.
3. Is this card asking what I want to know about the topic?
   • Maybe you’re asking the card in a way that’s difficult to remember, and that’s not even exactly what you want to know.
   • If so: Adjust your cards to better match what you want to know by adding new cards and/or editing the existing card and returning it to review.
4. Is this card non-critical?
   • If there’s no obvious problem with the card, consider whether you can’t just delete it and free up time for easier material. Some critical material won’t lend itself to deleting, but most material will.
   • If so: Delete the card and don’t look back.
5. Can I wait to learn this card?
   • If this card isn’t high-priority and you have other high-priority material to get through, you might want to focus on that other material first before deciding what to do with this card.
   • If so: Leave the card suspended and yourself in a state of ignorance until you have time to find a better way to learn the card.
6. Can I develop a mnemonic for this card?
   • If the card is critical, doesn’t have anything wrong with it, and you urgently need to know it, but you’ve still kept on forgetting it, do some work devising a good trick to remember it, and perhaps add other cards to help you remember the mnemonic.
   • If so: Develop the mnemonic, add a card to help you learn the mnemonic (or at a minimum, write the mnemonic on the back of the original card in case you forget it), and return the card to review.

If you’ve legitimately made a strong effort on all of these steps and somehow failed to apply any of them, you may choose to return the card to review even though it’s a leech. Be aware, however, that you’ll probably just keep forgetting the card if you do this. Leeches are flagged because it’s unlikely you’ll ever learn them properly unless you change tack. In Anki, the card will get marked as a leech again after another 4 lapses.

Is there an obvious reason I am frequently forgetting this card?

Often, you know exactly why you can’t remember the answer to a leeched question, you’ve just never gotten around to fixing it. Sometimes you’ll feel a glimmer of recognition when the “leech” bubble comes up: “Yeahhh, I never liked that question.” Other times, especially for beginners, the problem won’t be immediately obvious, but a little bit of careful thought will expose a straightforward issue with the question. Perhaps it’s unclear what the question is asking, it’s easily confused with a different question because of bad wording, or it does not follow the Minimum Information Principle. Or maybe you don’t understand some of the terms or concepts used on the card.

To fix: Make sure the card is precise and that you fully understand it. Add new cards to complete your understanding if necessary.

Can I shore up the question by adding additional cards?

This solution works for several different problems:

• You may be missing some basic context needed to understand the topic well enough to answer the question (Rule #1: Do not learn if you do not understand!). In this case, doing a little bit of research and adding additional cards to complete your understanding can rescue your card from leechdom.

• Leeches often occur when you don’t have a clear handle on the difference between two words or other similar items (the technical term for this is memory interference). You may regularly give the answer to one for the other, causing both of them to become leeches. But if you add another card asking yourself to distinguish between them, suddenly all three cards may become easy. This feels magical, but it often works. (Before doing this, though, be sure the problem isn’t caused by poorly worded, imprecise questions. If the questions could be improved, fix those rather than trying to paper it over with an extra card.)

• Sometimes you can extract a part of the question to make another question that’s useful on its own and also helps with your memory of the leech. For instance, I kept forgetting the German word halbkugelig, meaning “hemispheric”. 31 reviews and 8 lapses later, I realized that I didn’t immediately know that a Kugel is a sphere or ball (though I could work it out with a little bit of thought), and once I learned that on a separate card, the original card became dead easy.

To fix: Identify what aspect of the question you’re struggling with, learn more about that aspect, and create cards so you remember it.

Is this card asking what I want to know about the topic?

Perhaps you’re asking the question in a difficult way (for instance, as an enumeration), but you could recast it in an easier way, one which would be as good or better at testing precisely what you want to know.

To fix: Edit the card appropriately and put it back into review, or create new cards and delete this one, as necessary.

Is this card non-critical?

If there’s no obvious problem with the card – it’s just hard – consider whether you can’t chuck the card and reallocate your scarce study time to material you can learn more easily instead. Often it’s not worth continuing to hammer something into your head that just isn’t taking.

At first, many people find the idea of simply deleting difficult material to be strange or even absurd. To some degree, it may feel like dishonorably “giving up” as soon as we encounter resistance. But I suspect it’s mostly because our experience with organized studying is typically in an academic context, where we’re likely to be graded on our knowledge of a specific, ordained set of material. Thus, we feel that if we have created flashcards for the material, we’d better learn exactly that material.

It rarely works this way once you’re out of school, though – and indeed, it’s often not how it works even when you’re studying for an exam. The primary goal is usually volume: you want to learn as much useful information as possible in the amount of time you’re willing and able to spend studying. This means that if one card is taking up an unreasonable amount of your time, and by deleting it you can learn several other useful pieces of information, that’s probably a good trade, unless this card is unusually important.

Now here’s the math. SuperMemo’s theory page finds that removing the most difficult 10% of your material can improve your learning efficiency by up to 300% (see note at the end of this section for possible caveats). Think about that for a moment – by punting on the hardest tenth of your material, you can learn three times the amount of material in the same amount of time. Moreover, removing the most difficult material will make your study sessions more pleasant and keep you motivated. The hardest portion of the material had better be pretty darn important to make it worth keeping!

So if you weren’t able to easily fix the card, unless it’s critical, you should strongly consider axing it. Generally, a card might become critical in one of two ways:

• The particular piece of information the card contains is critical in itself (e.g., you’re certain you’re going to be tested on it, or it’s an important building block for other things you need to know). Ask yourself, “Will I be kicking myself for my stupid decision if I can’t remember this information later and know I deleted the card?” If so, that’s a critical card.
• The piece of information is an essential part of a larger whole that you want to memorize (for instance, a line of a poem).

If your card is critical, you’ll need to keep moving. I like to also tag the note as Critical – this way, if I am looking through leeches or otherwise-difficult cards again in the future, I’ll know that I should exclude this card from any mass deletions I might consider running.

Theoretical note: The theory page from which the 300% figure cited above comes discusses a “generic” spaced-repetition algorithm, but its assumptions appear to be targeted at the SM-11 algorithm, which is not the same as Anki’s algorithm and which I don’t know enough about to confidently say all the assumptions transfer. (If anyone knows the math better than me and can speak to this, let me know. I’m more of a practitioner than a theoretician on spaced repetition, particularly when it comes to algorithms I don’t use.) So it’s probably a bad idea to assume that this figure will transfer exactly to Anki. But in any case, it’s a statistical estimate, not an exact number that will apply to every collection. And setting aside the precise numbers, it is logically unassailable that in any functional spaced-repetition algorithm, deleting the hardest material and adding more average-difficulty material will allow you to learn more material, because the hardest material is by definition repeated much more frequently and consequently requires much more review time than the average material (unless you’re willing to make the strange assertion that harder material takes less time to study even though it gets repeated more frequently!) Any spaced-repetition user with some review history can verify this with a cursory look at the time they’ve spent reviewing the hardest cards in their collection compared to the average cards.

Can I wait to learn this card?

If editing a card doesn’t seem particularly likely to help and you don’t feel comfortable deleting it, it may make sense to leave the card suspended and come back to it later. This approach is especially useful if you’re trying to learn a large amount of information before a looming deadline. For instance, suppose you’re trying to learn 2,000 items for a professional certification exam you’re taking in three months. You probably don’t know for sure what your workload will look like over those three months and how much time you’ll have to study, so if you find leeches, it makes sense to set them aside so you can keep crunching through the rest of the material as quickly as possible. If you work your way through all the other items with time to spare, then you can unsuspend the leeches and decide whether to relearn them, reformulate them, or just delete them and accept that you might miss those questions if they come up on the exam. If you don’t make it to the end, then you’ll have learned more material and you’ll presumably do better on the exam than if you spent your time working on the leeches.

Can I develop a mnemonic for this card?

If you’ve exhausted your options for improving the card and you can’t afford to put it aside temporarily or permanently, it’s time to at least develop an artificial device to help yourself remember it. One mnemonic technique or another can be applied to almost anything, so if you don’t know how, it’s time to learn some good mnemonic techniques! More to come in this series, but in the meantime, plenty of resources are available on the web.

I like to create a separate card asking about the mnemonic, then edit the original card to explain the mnemonic on the front and ask for the answer based on the mnemonic. For instance, suppose you want to use the acronym “I Don’t Pee Like My Aunt Leona” to remember an enumeration of the musical modes. (My high-school orchestra director shared this mnemonic with us, and after hearing it once I have never forgotten it! Lest your imagination run a bit too wild, evidently Aunt Leona had a permanent urinary catheter.) One card would say on the front, “What is my mnemonic for the order of the musical modes?”, with the phrase on the back, and a second card would say on the front, “What are the musical modes in order? (‘I Don’t Pee Like My Aunt Leona’)”, with “Ionian – Dorian – Phrygian – Lydian – Mixolydian – Aeolian – Lochrian” on the back.

This method makes it possible to test your recollection of the mnemonic and your ability to use the mnemonic separately. Not only does that hew closer to the Minimum Information Principle and make scheduling more efficient, it also means that if your mnemonic is poor and you continue forgetting, you have more information about which part of it is problematic. You additionally introduce a small amount of redundancy – you’ll likely activate some of the memory traces for answering the question when recalling the mnemonic, and some for the mnemonic when answering the question, giving yourself a little extra passive repetition on a card you know to be difficult.

In practice, if you apply the previous five steps well, you’ll rarely get to this point – though this is not to say you can’t or shouldn’t consciously apply mnemonic devices to help you remember cards that don’t become leeches, just that you will rarely be left with no other option.

Dealing with leeches in Anki

Now, here are some tips for handling the mechanics of leech management. First of all, you need to look for leeches periodically and decide what to do with them; you find leeches by searching for tag:leech in the browser (or clicking the “leech” tag in the sidebar). I like to be proactive and do this every week or two, but because Anki suspends leeches once it identifies them, your study efficiency won’t suffer if you do it less often – you’ll just be without the knowledge on those particular cards for a bit longer.

Once you’ve found the leeches and decided what action to take on them, here’s how to take that action.

Note: In the keyboard shortcuts listed here, replace Ctrl with Command if you’re using a Mac.

• Delete: If you’ve decided to delete a difficult card, just press Ctrl-Delete or right-click on it and choose Delete.
• Edit a card: You can directly edit the contents of a card in the bottom pane of the browser.
• Add cards: If you need to add new cards from the browser, press Ctrl-E or choose Notes > Add Notes from the menu.

• Return to review: Press Ctrl-Alt-R or right-click and choose Reschedule, then press Enter to choose the default of “Place at end of new card queue.” The card will be scheduled for reintroduction as if it were new, though it will retain knowledge of its previous life in its review history if you look at its card info (Ctrl-Shift-I).

  Some people would just unsuspend the card (Ctrl-J, or right-click and Toggle Suspend – no, I don’t know why it’s J instead of the completely unused Ctrl-S), but I usually prefer to give the card a clean slate with a 250% ease, as we’ve hopefully made it so much easier to remember now that it makes sense to schedule it fresh. However, if you don’t think you’ve changed the card enough to merit resetting its scheduling – this might be especially true if you only developed a mnemonic without changing the card itself – you might prefer to just unsuspend it.

When you clean up your leeches, you’ll also want to remove the leech tag from the cards so that you don’t later try to improve cards you’ve already fixed! The easiest way to do this is to work your way through all of the leeches, then mass-select all of them and press Ctrl-Shift-D or right-click and choose Remove Tags, then enter leech in the box and click OK.

Note: The methods for deleting, editing, adding, and adjusting the tags of “cards” I described above actually delete, edit, add, and tag notes, not cards. If you’re just starting with Anki, the distinction may not be important, but if you start to use more complex note types that generate more than one card, you need to be aware that these actions may affect multiple cards. To get a grip on the relationship between cards, notes, and note types, review the Key Concepts section of Anki’s manual.