read_fwf: encoding is ignored when memory_map is True

df = pd.read_fwf(
                fname, 
                names=columns, 
                header=None, 
                widths=col_len, 
                iterator=True,
                chunksize=chunk_size,
                encoding="iso8859_1",
                memory_map=True,
            )

Problem description

I need to read a file that contains German Umlaute (äüö).
pd.read_fwf() works great as long as memory_map is not True.
When enabling memory_map (as in the example above), a Unicode exception is thrown at the first position of a German Umlaut. I can toggle the exception by enabling/disabling memory_map.

Expected Output

option 1) Preferably encoding is supported with memory_map.
option 2) If for technical reasons this combination is not possible, an exception pointing out that this combination is not allowed,
option 3) If it is not easy to detect whether the combination of memory_map and encoding is suitable (mihgt be it does work on Linux or for specific encodings, system defaults, ..), then a warning might be an option
option 4) if nothing works, then please update the documentation of read_csv, read_fwf, etc.. pointing at possible problems under specific circumstances.

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.6.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: AMD64 Family 21 Model 101 Stepping 1, AuthenticAMD
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.23.4
pytest: 3.8.2
pip: 18.1
setuptools: 40.4.3
Cython: None
numpy: 1.15.2
scipy: None
pyarrow: 0.11.0
xarray: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.7.3
pytz: 2018.5
blosc: None
bottleneck: None
tables: 3.4.4
numexpr: 2.6.8
feather: None
matplotlib: None
openpyxl: 2.5.8
xlrd: 1.1.0
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: 1.2.12
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

[TomAugspurger]

Thanks for the report.

Any interest in working on a fix? If not, it'd be valuable if you could write a test case (write some iso8859_1-encoded data to a BytesIO buffer, and read the back in with read_fwf).

Test case:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import pandas as pd
from io import BytesIO
import tempfile

def test_encoding_mmap(memory_map):
    encoding = "iso8859_1"
    data = " 1 A Ä 2\n"
    data = data.encode(encoding)
    # with tempfile.NamedTemporaryFile(mode="wb") as fp:  # On windows doesn't allow me to read it again
    fname = tempfile.mktemp()
    with open(fname, "wb") as fp:
        fp.write(data)

    widths = [2, 2, 2, 2]
    try:
        return pd.read_fwf(fp.name, header=None, widths=widths, encoding=encoding, memory_map=memory_map)
    finally:
        os.remove(fname)

df = test_encoding_mmap(False)
assert len(df) == 1

try:
    df = test_encoding_mmap(True)
    assert len(df) == 1
except:
    assert False, "Ups. Shouldn't happen"