Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add native Windows support #1252

Closed
hanxiao opened this issue Nov 9, 2020 · 5 comments · Fixed by #3290
Closed

add native Windows support #1252

hanxiao opened this issue Nov 9, 2020 · 5 comments · Fixed by #3290
Labels
status/blocked-help-needed This issue is blocked and needs support type/feature-request This issue describes a new feature or behaviour a user or users desires.

Comments

@hanxiao
Copy link
Member

hanxiao commented Nov 9, 2020

Describe the feature

Jina can not run on Windows natively, please add your issues/observations here. We need community help on this one.

There are several knowns issues,

  1. uvloop does not support Windows. uvloop is used in jina by default for boosting the performances, but it is known in the uvloop community. windows support? MagicStack/uvloop#14.
  2. SystemLogHandler uses TCP on unix while UDP on Windows.
  3. Multiprocessing in Windows does not work for jina.
Traceback (most recent call last): 
  File "C:\Users\magic\AppData\Local\Programs\Python\Python38-32\Scripts\jina-script.py", line 33, in <module> 
    sys.exit(load_entry_point('jina', 'console_scripts', 'jina')()) 
  File "c:\users\magic\codes\github\jina-master\cli\__init__.py", line 90, in main 
    getattr(api, args.cli.replace('-', '_'))(args) 
  File "c:\users\magic\codes\github\jina-master\cli\api.py", line 8, in pod 
    with Pod(args) as p: 
  File "c:\users\magic\codes\github\jina-master\jina\peapods\pod.py", line 254, in __enter__ 
    return self.start() 
  File "c:\users\magic\codes\github\jina-master\jina\peapods\pod.py", line 224, in start 
    self.enter_context(p) 
  File "c:\users\magic\appdata\local\programs\python\python38-32\lib\contextlib.py", line 425, in enter_context 
    result = _cm_type.__enter__(cm) 
  File "c:\users\magic\codes\github\jina-master\jina\peapods\pea.py", line 424, in __enter__ 
    return self.start() 
  File "c:\users\magic\codes\github\jina-master\jina\peapods\pea.py", line 398, in start 
    super().start() 
  File "c:\users\magic\appdata\local\programs\python\python38-32\lib\multiprocessing\process.py", line 121, in start 
    self._popen = self._Popen(self) 
  File "c:\users\magic\appdata\local\programs\python\python38-32\lib\multiprocessing\context.py", line 224, in _Popen 
    return _default_context.get_context().Process._Popen(process_obj) 
  File "c:\users\magic\appdata\local\programs\python\python38-32\lib\multiprocessing\context.py", line 327, in _Popen 
    return Popen(process_obj) 
  File "c:\users\magic\appdata\local\programs\python\python38-32\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__ 
    reduction.dump(process_obj, to_child) 
  File "c:\users\magic\appdata\local\programs\python\python38-32\lib\multiprocessing\reduction.py", line 60, in dump 
    ForkingPickler(file, protocol).dump(obj) 
_pickle.PicklingError: Can't pickle <class 'jina.peapods.pea.BasePea'>: it's not the same object as jina.peapods.pea.BasePea 
PS C:\Users\magic\Codes\github\tmp> Traceback (most recent call last): 
  File "<string>", line 1, in <module> 
  File "c:\users\magic\appdata\local\programs\python\python38-32\lib\multiprocessing\spawn.py", line 102, in spawn_main 
    source_process = _winapi.OpenProcess( 
OSError: [WinError 87] 

Your proposal

  1. uvloop is installed by default when running pip install "jina[core]". One needs to figure out a way to avoid install uvloop in the extra-requirements.txt

  2. Add the logic for processing windows.
    https://github.com/jina-ai/jina/blob/master/jina/logging/logger.py#L155
    https://github.com/jina-ai/jina/blob/master/jina/logging/logger.py#L164

  3. Need more time to look into the issue.


Environment

jina                          0.7.5
jina-proto                    0.0.75
jina-vcs-tag                  (unset)
libzmq                        4.3.2
pyzmq                         1.18.1
protobuf                      3.13.0
proto-backend                 cpp
grpcio                        1.27.2
ruamel.yaml                   0.16.10
python                        3.7.5
platform                      Darwin
platform-release              19.5.0
platform-version              Darwin Kernel Version 19.5.0: Tue May 26 20:41:44 PDT 2020; root:xnu-6153.121.2~2/RELEASE_X86_64
architecture                  x86_64
processor                     i386
jina-resources                /Users/nanwang/Codes/jina-ai/jina/jina/resources
JINA_ARRAY_QUANT              (unset)
JINA_BINARY_DELIMITER         (unset)
JINA_CONTRIB_MODULE           (unset)
JINA_CONTRIB_MODULE_IS_LOADING(unset)
JINA_CONTROL_PORT             (unset)
JINA_DB_COLLECTION            (unset)
JINA_DB_HOSTNAME              (unset)
JINA_DB_NAME                  (unset)
JINA_DB_PASSWORD              (unset)
JINA_DB_USERNAME              (unset)
JINA_DEFAULT_HOST             (unset)
JINA_DISABLE_UVLOOP           (unset)
JINA_EXECUTOR_WORKDIR         (unset)
JINA_FULL_CLI                 (unset)
JINA_IPC_SOCK_TMP             (unset)
JINA_LOG_CONFIG               (unset)
JINA_LOG_NO_COLOR             (unset)
JINA_POD_NAME                 (unset)
JINA_PROFILING                (unset)
JINA_RANDOM_PORTS             (unset)
JINA_SOCKET_HWM               (unset)
JINA_TEST_GPU                 (unset)
JINA_TEST_PRETRAINED          (unset)
JINA_VCS_VERSION              (unset)
JINA_WARN_UNNAMED             (unset)

Screenshots

@hanxiao hanxiao added type/feature-request This issue describes a new feature or behaviour a user or users desires. status/blocked-help-needed This issue is blocked and needs support labels Nov 9, 2020
@hanxiao
Copy link
Member Author

hanxiao commented Nov 9, 2020

@nan-wang @deepankarm please post your Windows running errors here

@nan-wang
Copy link
Member

@nan-wang @deepankarm please post your Windows running errors here

Updated the issue accordingly.

@jina-bot
Copy link
Member

jina-bot commented Dec 1, 2020

This issue is stale because it has been open 20 days with no activity. Remove stale label or comment or this will be closed in 4 days

@jina-bot jina-bot added the Stale label Dec 1, 2020
@jina-bot jina-bot closed this as completed Dec 5, 2020
@yogeshhk
Copy link

RuntimeError: uvloop does not support Windows at the moment

@nan-wang
Copy link
Member

nan-wang commented Jan 4, 2021

@yogeshhk hi, there. Thanks for your interests. The native Windows support is not there yet. You can try Jina with WSL. https://docs.jina.ai/chapters/install/os/on-wsl.html

@cristianmtr cristianmtr reopened this Aug 27, 2021
@cristianmtr cristianmtr removed the Stale label Sep 13, 2021
@cristianmtr cristianmtr reopened this Sep 13, 2021
@deepankarm deepankarm linked a pull request Sep 13, 2021 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status/blocked-help-needed This issue is blocked and needs support type/feature-request This issue describes a new feature or behaviour a user or users desires.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants