Leo University

This is a backup of the online LeoU issues.

Lesson 1: Think process, not knowledge

Programming is much easier than you might think.

Many of Leo’s most important devs made major contributions to Leo without asking me a single question. Do they have super powers? No, they don’t.

But they do know the secret of programming: Simple processes, endlessly repeated, suffice to find, understand and change *any* software. And Leo simplifies these techniques even further. Here they are…

Basic skill/process 1: Familiarize yourself with the code

For any project you want to know what files exist. When studying other people’s code I use a recursive import script https://github.com/leo-editor/leo-editor/issues/846 to create a tree of files. The goal is just to have a vague understanding of what files are involved. Don’t even think about remember details.

It’s even easier with Leo. LeoPy.py contains Leo’s sources. Peruse this outline, seeing what @file nodes it contains.

Don’t make a big deal about this step. It should just take 5 or 10 minutes. Look only at files that interest you. In those files you will probably want to look briefly at the top-level classes in the files that interest you. You will likely repeat this step many times, adding a little to your understanding each time.

Basic skill/process 2: Learn what to search for

We devs often want to change how commands work. You can usually find the top-level code for a command simply by searching for the command’s name, in single quotes.

To find the definition of a method, function or class, simply control-click on it. That will start a search for def x and class x. If that doesn’t work, a cff (clone-find-all-flattened) search on a name will find all uses of that name in the present outline.

Basic skill/process 3: Look for helpers in child nodes

With Leo outlines it’s natural to put a method’s helpers in child nodes. I usually forget implementation details, but the children of a node usually contain most of the details.

Basic skill/process 4: Use clones to organize and remember your work

When working on a project I make create an organizer node as the last top-level node of the outline. This node will typically contain a link to the github issue. I make clones of all relevant nodes and move them so they are clones of the organizer node. For complex projects I may create subsidiary organizer nodes, like “changed”, “reference”, etc.

When a project is done (or suspended), I move the project’s organizer node so it is the last child of another organizer node called “Recent Code”. This hides all work except the what I am, this minute, working on. This is important: it hides details and lessens distractions.

Summary

The easy processes just described form the basis for programming workflow.

Expect to repeat these processes many times. Don’t worry if not everything is clear at first.

Lesson 2: Learn Leo’s debugging tools

Here are the basics of debugging with Leo:

g.trace

g.trace(*args, **kwargs) prints its arguments to the console. I use g.trace all throughout Leo’s code base.

g.pdb

g.pdb() invokes python’s pdb debugger, https://docs.python.org/3/library/pdb.html, adapted to Leo’s Qt gui. I use g.pdb when g.trace doesn’t tell me enough.

Lesson 3: Running unit tests

​All devs must run all unit tests before commit anything. Here’s how:

cd <path to>leo-editor
python -m unittest %*

Important: Some unit tests may fail on your environment. Don’t worry about them. You need only ensure that your new code causes no new unit tests to fail. If in doubt, ask on leo-editor.

Lesson 4: Use issues as an programming notebook

I recommend using github issues to track your work. This is especially true for Leo devs. You want other people to be able to follow and comment on your work.

The convention is to have the first comment for any issue contain an up-to-date summary of the status of the issue. I have found that it is useful to continually edit the first comment.

Lesson 5: c.recursiveImport imports files recursively

c.recursiveImport is a wrapper on the RecursiveImportController class.​ ​You can find the related code by doing a regex cff (ignoring case) on ​ ​“recursive?import”.

Here is how to use c.recursiveImport:

c.recursiveImport(
    dir_ = << path to root directory >>,
        # You must expand ~ yourself
    kind = '@clean', # '@clean', @nosent','@auto','@file'
    recursive = True,
    safe_at_file = False,
    theTypes = ['.py',],
)

I like to import to @clean and set safe_at_file = False, but before doing that I create a git repo for the to-be-imported directory and check in all the .py files. That way I can track any changes that get made, and revert them if need be.

Lesson 6: How devs should use git branches

Marcel Franke showed how to use the recommended way of using git branches: https://groups.google.com/d/msg/leo-editor/p_6ljMQL660/57Z8lYuQAAAJ

Here are his remarks, slightly edited:

  • master is stable.

  • devel is “mostly” stable.

  • feature branches off devel can be wildly unstable.

  • release branches are candidates for master, so should be stable.

Everything new comes from devel and goes to devel. Devs must run unit tests before pushing to devel, so devel should usually be stable.

All experimental code should reside in a feature branch.

At all times, anyone can pull stable code from master.

Only the maintainer (EKR) is supposed to work on release or master, so nobody needs to hold back or knows what is going on in them. At all times everyone can happily hack on devel, or better, their personal feature branches.

To prepare a new release, the maintainer branches devel to a new branch, say 6.x.y. The maintainer runs tests on it, and makes any last-minute fixes. To publish the release, the maintainer merges the release to master and to devel, then optionally deletes the release branch.

This visualization made it all clear to me: https://blog.seibert-media.net/wp-content/uploads/2014/03/Gitflow-Workflow-3.png

Lesson 7: How to make complex changes to a code base

This lesson discusses the process of making arbitrarily complex changes throughout a code base.

It’s an elaboration of Lesson 1: Think process, not knowledge, but it’s worth reading because it contains helpful hints about tackling any big project. I used this general method when making Leo’s code base work with Python 3 as well as Python 2.

The grand summary:

  1. Study the code first with clone-find commands.

  2. Let python crashes & failed unit tests guide you.

In detail:

  1. Create a new git branch for the work.

  2. Create a global master switch (a boolean) that separates and marks the old and new versions of the code. The switch selects the new code if True and the old code otherwise.

  • All unit tests should continue to pass when the switch selects legacy code.

  • All pushes should have the legacy code selected until every works.

These two requirements make it safe to pause work on the project.

  1. Make sure you understand what needs to be done. Use the clone find commands to answer any and all questions about the code and how it should be changed.

  2. Make the fundamental changes required, without worrying about the consequences.

For the “keys” branch I changed the representation of settings so that they are NewLeoKey objects.

  1. With the switch in the “new” position, run a test copy of Leo.

Because of step 4, we expect crashes. Fix each as they happen, separating the old and new code with a test on the switch. Repeat this step until no crashes happen.

When this phase works, set the switch to “legacy”, re-run all unit tests, and commit the code.

  1. For the “keys” branch, getting through startup is only half the task. When I type a key I expect more crashes. I’ll follow step 5 until no crashes happen when I type various key combination.

Again, I’ll set the switch to “legacy”, re-run all unit tests and commit the code.

  1. When normal operation appears to work with the switch in the “new” position, run all unit tests and fix any crashes or unit tests failures.

Now you can commit the code with the switch in the “new” position.

  1. At this point you can start eating your own dog food. When satisfied that the code is stable enough, you can do the following:

  • Remove the switch and all code disabled when the switch is in the “legacy” position.

  • Run all unit tests.

  • Continue to eat your own dog food.

  • Commit the branch to “devel” when all seems well.

That’s the process. It’s extremely effective. You don’t have to remember your changes because a cff on the switch should yield all changed code. Naturally, you can make clones of nodes containing the most important changes.

LeoU: Suggestions for theme developers

I recommend using the organization of leo/themes/EKRDark.leo as a guide. Note: Syntax coloring settings are required because they are not used in the css.

To avoid confusion, it’s best if theme .leo files contain no unbound settings. This will ensure that the appearance of a theme does not unwittingly depend on settings in myLeoSettings.leo.

Important: Theme devs should set the following theme description settings:

@string theme_name   # crucial
@bool color_theme_is_dark
@string color_theme = ekr_dark

These settings ensure that relative urls (including “:/” syntax are resolved properly in css, and that icons are found in the proper directories.

Icons boxes should have constant width. Otherwise, the variable indentation of the rest of the headline appears to represent variable outline structure, which is highly confusing.

LeoU: About Leo’s architecture

By the term architecture I mean the design of Leo’s modules and classes.

Information hiding

Leo’s design is based on the thinking of Glenford Meyers. Let’s use the word component to mean either a Python module or class.

We devs must be able to change the implementation of a component without affecting any other code. This has been true throughout Leo’s 20+ years of history. This is the foundation on which Leo is built.

Principle 1: Each class should concern itself with one, easily defined “area of concern”.

Modules should contain related classes. All components should have short, simple descriptions, no matter how complex they are internally.

Principle 2: Components should know as little as possible about the internal details of other components.

Components have well-defined boundaries. The inside of a component contains the implementation. The outside of a component is its API.

Changes to the API can break code. However, knowing about the existence of a component’s high-level methods does not (usually) blur the boundaries of a component!

In most cases, components should know nothing about the ivars and low-level methods. However, components can freely use the so-called official ivars of other classes. These official ivars are highly unlikely ever to change. And they are useful. We wouldn’t want to require long-winded alternatives to, say, c.frame.body.wrapper.

Class design: inheritance vs instantiation

Leo uses the “has-a” pattern, rather than the “is-a” class pattern. That is, Leo gains access to the capabilities of other classes by instantiating the other classes rather than being subclasses of other classes. The “has-a” pattern is a personal preference of mine, and is now pretty much baked into Leo.

The “has-a” pattern completely insulates classes from the internal details of other classes. There is no need to worry about clashing ivars, or _ ivars, etc. Otoh, the “has-a” pattern results in more complex startup code. Imo, that “extra” complexity is well worthwhile.

LeoU: About Leo’s Position and VNode classes

From this post: https://groups.google.com/forum/#!topic/leo-editor/SjHfXepxiTo Not everyone agrees with everything I say here.

This LeoU discussion discusses the present code base only. #883 (a low priority item) discusses ways in which the tree of vnodes could be decoupled from c, the Commands instance.

The Position class

The Position class concerns itself only with traversing the tree. The position class contains no node-related data. In order to insert top-level nodes, methods of the Position class gets access to c.hiddenRootNode via p.v.context.

The VNode class

The VNode class contains all data in an outline, as well as crucial, low-level operations on those data. There is no other place to store node-related data.

The VNode class is one of the foundations of Leo’s scripting. As a result, none of the ivars or methods of the VNode class can change in any way. That is, all of the ivars of the VNode class are “official”. Scripts can, and do, access these ivars directly. Perhaps the “context” kwarg of VNode.__init__ should just be called “c”, but that can’t be done now.

Theoretically, it would be possible to add new kwargs to VNode methods, but that is extremely unlikely. It would also be possible to add new ivars to the VNode class, but uA’s make that unnecessary.

The properties of the VNode class, v.b, v.h, v.gnx, v.u, allow higher-level access to various VNode ivars, but in some cases Leo’s code accesses the corresponding ivars directly, for greater performance.

Interaction with other classes

The v.context ivar is controversial. In fact, the v.context ivar is completely benign. It has never caused problems, and it never will. It does not, in any significant way, make the VNode class dependent on any other class. It has no harmful architectural consequences.

Yes, the FileCommands class must know about gnx’s, but the converse is not true. The VNode class knows nothing about how other classes use gnx’s. The FileCommands class creates VNode instances without knowing anything about low-level VNode methods.

The VNode class is, in fact, an excellent example of information hiding. The boundary between the VNode class is well defined and well enforced.

Possible improvements

  1. A new c.new_vnode method could help instantiate VNodes:

def new_vnode(self, gnx=None):
    c = self
    return leoNodes.VNode(context=c, gnx=gnx)

The (small) advantage is that clients of c.new_vnode(gnx) would no longer need to import leo.core.leoNodes.

  1. The v.saveCursorAndScroll and v.restoreCursorAndScroll could be moved elsewhere. At present, these methods cause no problems, but moving them would be part of #883.