Improve the use of __doc__ in pydoc

[serhiy-storchaka]

BPO	40257
Nosy	@gvanrossum, @terryjreedy, @mdickinson, @ncoghlan, @ambv, @serhiy-storchaka, @vedgar, @ilevkivskyi, @tacaswell, @Carreau, @eamanu, @tirkarthi
PRs	bpo-40257: Output object's own docstring in pydoc #19479 bpo-40257: Improve help for the typing module #19546 bpo-40257: Tweak docstrings for special generic aliases. #20022 bpo-40257: Revert changes to inspect.getdoc() #20073 bpo-29940: Add follow_wrapped option to help() #22390

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-04-11.20:54:38.793>
labels = ['type-feature', 'library', '3.9']
title = 'Improve the use of __doc__ in pydoc'
updated_at = <Date 2021-11-04.14:25:10.422>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2021-11-04.14:25:10.422>
actor = 'erlendaasland'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-04-11.20:54:38.793>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 40257
keywords = ['patch']
message_count = 29.0
messages = ['366220', '366225', '366228', '366230', '366370', '366371', '366376', '366547', '366715', '366716', '368584', '368606', '368607', '368608', '368609', '368611', '368612', '368618', '368632', '368673', '368738', '368790', '368792', '368797', '368798', '368990', '369279', '369290', '369307']
nosy_count = 12.0
nosy_names = ['gvanrossum', 'terry.reedy', 'mark.dickinson', 'ncoghlan', 'lukasz.langa', 'serhiy.storchaka', 'veky', 'levkivskyi', 'tcaswell', 'mbussonn', 'eamanu', 'xtreak']
pr_nums = ['19479', '19546', '20022', '20073', '22390']
priority = 'high'
resolution = 'remind'
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue40257'
versions = ['Python 3.9']

[serhiy-storchaka]

Currently pydoc outputs __doc__ for classes, functions, methods, properties, etc (using inspect.getdoc()). If the object itself does not have non-empty __doc__, it searches non-empty __doc__ in the class parenthesis (if the object is a class) or in the corresponding overloaded members of the class to which the object (method, property, etc) belongs.

There are several problems with this.

1. Using the docstring of a parent class is misleading in most classes, especially if it is a base or abstract class (like object, Exception, Mapping).

2. If the object does not have the __doc__ attribute, it inherits it from its class, so inspect.getdoc(1) returns the same as inspect.getdoc(int).

3. If the object has own docstring, but is not a class or function, it will be output in the section DATA without a docstring.

The following PR fixes these issues.

1. Docstrings for classes are not inherited. It is better to not output a docstring than output the wrong one.

2. inspect.getdoc() returns the object's own docstring.

3. Docstrings are always output for object with a docstring. See for example help(typing).

In future issues I'll make help(typing) even more informative.

[vedgar]

I don't agree with 1. I use that feature a lot, I write a base class which my students must subclass to their liking, but they still expect that help(TheirClass) will give them the documentation they need.

I agree that in _some_ cases it is not helpful (but even when the base is abstract, it might be helpful). How about: we keep the current behavior, but make it clear that the docstring applies to a superclass? It might be subtle, as just changing the first line of help() output (currently it says "Help on class Derived in module ...", change it to "Help on class Base in module ..."), or write a longer message such as "Documentation for Derived not found, showing the documentation for Base". But just removing it in all cases is really a wrong thing to do.

[ilevkivskyi]

FWIW I like the idea. There are many objects in typing module that are not classes, it would be great to display docs for them.

[serhiy-storchaka]

Inheritance of docstrings was added in bpo-15582. It works good for class members, but I now realized that doing it for class itself was a mistake. For example:

>>> import wave
>>> help(wave.Error)
Help on class Error in module wave:

class Error(builtins.Exception)
 |  Common base class for all non-exit exceptions.
 |  
 |  Method resolution order:
 |      Error
 |      builtins.Exception
 |      builtins.BaseException
 |      builtins.object
 |  
...

I fixed many similar issues by adding docstrings to classes, but there are even more exception classes and other classes in the stdlib for which help() gives incorrect description. I don't remember a single case when inheritance of the docstring was helpful.

Note that help() outputs the list of base class right after the docstring, so it is not hard to give an additional information, especially in interactive browser mode. If you want to inherit a docstring, you can do it explicitly:

    __doc__ = BaseClass.__doc__

[vedgar]

Ok, I get what you're saying. But if someone writes

    class B(A):
      # no docstring at all
      ...

    help(B)

they'll still get other elements of current help? Particularly, "Methods inherited from A" (with their docstrings)?

[serhiy-storchaka]

Yes, of course. And if it overrides some methods, but do not specify doctrings for new methods, they will be inherited from the parent class.

class A:
    """Base class"""
    def foo(self): """Some docstring"""
    def bar(self): """Other docstring"""

class B(A):
    def foo(self): pass

help(B)

Help on class B in module __main__:

class B(A)
 |  Method resolution order:
 |      B
 |      A
 |      builtins.object
 |  
 |  Methods defined here:
 |  
 |  foo(self)
 |      Some docstring
 |  
 |  

---

| Methods inherited from A:
|
| bar(self)
| Other docstring
|
...

[vedgar]

Then I'm fine with it. Thanks.

[serhiy-storchaka]

New changeset fbf2786 by Serhiy Storchaka in branch 'master':
bpo-40257: Output object's own docstring in pydoc (GH-19479)
fbf2786

[serhiy-storchaka]

New changeset 7e64414 by Serhiy Storchaka in branch 'master':
bpo-40257: Improve help for the typing module (GH-19546)
7e64414