query_constructor throws OutputParserException is query is null

[JanHorcicka]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Example Code

See description.

Description

I am using SelfQueryRetriever. For a response JSON that contains null query, for example:

json
    {
        "query": null,
        "filter": ...
    }

The output parser throws OutputParserException at line 51.

OutputParserException('Parsing text\njson\n{\n "query": null,\n "filter": "eq(\\"kategorie\\", \\"Pravo\\")"\n}\n\n raised following error:\nobject of type 'NoneType' has no len()')Traceback (most recent call last):

File "/home/MetaExponential/.local/lib/python3.10/site-packages/langchain/chains/query_constructor/base.py", line 51, in parse
if len(parsed["query"]) == 0:

System Info

absl-py==2.0.0
ai21==1.2.8
aioboto3==12.0.0
aiobotocore==2.7.0
aiohttp==3.8.4
aioitertools==0.11.0
aiosignal==1.3.1
altgraph @ file:///AppleInternal/Library/BuildRoots/9dd5efe2-7fad-11ee-b588-aa530c46a9ea/Library/Caches/com.apple.xbs/Sources/python3/altgraph-0.17.2-py2.py3-none-any.whl
annotated-types==0.6.0
annoy==1.17.3
antlr4-python3-runtime==4.9.3
anyio==3.7.1
argilla==1.7.0
astunparse==1.6.3
async-generator==1.10
async-timeout==4.0.2
attrs==23.1.0
Babel==2.12.1
backoff==2.2.1
bcrypt==4.0.1
beautifulsoup4==4.12.2
blinker==1.6.2
boto3==1.28.64
botocore==1.31.64
build==0.10.0
CacheControl==0.12.11
cachetools==5.3.1
camel-converter==3.1.0
certifi==2022.12.7
cffi==1.15.1
cfgv==3.3.1
chardet==5.2.0
charset-normalizer==3.1.0
Chroma==0.2.0
chroma-hnswlib==0.7.3
chromadb==0.4.13
cleo==2.0.1
click==8.1.7
clickhouse-connect==0.6.18
CoffeeScript==2.0.3
cohere==4.31
coloredlogs==15.0.1
commonmark==0.9.1
contourpy==1.0.7
crashtest==0.4.1
cryptography==40.0.2
cssselect==1.2.0
cycler==0.11.0
dataclasses-json==0.5.7
datasets==2.12.0
decorator==5.1.1
Deprecated==1.2.13
deprecation==2.1.0
dill==0.3.7
distlib==0.3.6
distro==1.8.0
dnspython==2.3.0
docutils==0.19
duckdb==0.7.1
dulwich==0.21.5
effdet==0.3.0
elastic-transport==8.10.0
elasticsearch==7.13.4
et-xmlfile==1.1.0
exceptiongroup==1.1.1
facebook-sdk==3.1.0
facebooktoken==0.0.1
faiss-cpu==1.7.4
fastapi==0.103.2
fastavro==1.8.2
feedfinder2==0.0.4
feedparser==6.0.11
filelock==3.12.0
Flask==2.3.2
Flask-Cors==4.0.0
Flask-Limiter==3.4.1
Flask-Mail==0.9.1
flatbuffers==23.5.26
fonttools==4.39.4
frozenlist==1.3.3
fsspec==2023.6.0
future @ file:///AppleInternal/Library/BuildRoots/9dd5efe2-7fad-11ee-b588-aa530c46a9ea/Library/Caches/com.apple.xbs/Sources/python3/future-0.18.2-py3-none-any.whl
fuzzywuzzy==0.18.0
gast==0.5.4
google-ai-generativelanguage==0.4.0
google-api-core==2.12.0
google-auth==2.23.3
google-auth-oauthlib==1.0.0
google-cloud-aiplatform==1.38.1
google-cloud-bigquery==3.12.0
google-cloud-core==2.3.3
google-cloud-resource-manager==1.10.4
google-cloud-storage==2.12.0
google-crc32c==1.5.0
google-generativeai==0.3.2
google-pasta==0.2.0
google-resumable-media==2.6.0
googleapis-common-protos==1.56.4
grpc-gateway-protoc-gen-openapiv2==0.1.0
grpc-google-iam-v1==0.12.6
grpcio==1.59.0
grpcio-status==1.59.0
grpcio-tools==1.59.0
h11==0.14.0
h2==4.1.0
h5py==3.10.0
hnswlib==0.7.0
hpack==4.0.0
html5lib==1.1
httpcore==0.16.3
httptools==0.5.0
httpx==0.23.3
huggingface-hub==0.14.1
humanfriendly==10.0
humbug==0.3.2
hyperframe==6.0.1
identify==2.5.23
idna==3.4
importlib-metadata==6.6.0
importlib-resources==5.12.0
iniconfig==2.0.0
install==1.3.5
installer==0.7.0
iopath==0.1.10
itsdangerous==2.1.2
jaraco.classes==3.2.3
jieba3k==0.35.1
Jinja2==3.1.2
jmespath==1.0.1
joblib==1.2.0
jq==1.6.0
jsonpatch==1.33
jsonpointer==2.4
jsonschema==4.17.3
jwt==1.3.1
keras==2.14.0
keyring==23.13.1
kiwisolver==1.4.4
lancedb==0.3.2
langchain==0.1.0
langchain-community==0.0.11
langchain-core==0.1.9
langchain-google-genai==0.0.5
langsmith==0.0.77
lark==1.1.7
layoutparser==0.3.4
Levenshtein==0.23.0
libclang==16.0.6
libdeeplake==0.0.84
limits==3.5.0
llama-cpp-python==0.1.39
lockfile==0.12.2
loguru==0.7.0
lxml==4.9.2
lz4==4.3.2
macholib @ file:///AppleInternal/Library/BuildRoots/9dd5efe2-7fad-11ee-b588-aa530c46a9ea/Library/Caches/com.apple.xbs/Sources/python3/macholib-1.15.2-py2.py3-none-any.whl
Mako==1.2.4
Markdown==3.4.3
markdown2==2.4.8
MarkupSafe==2.1.2
marshmallow==3.19.0
marshmallow-enum==1.5.1
matplotlib==3.7.1
meilisearch==0.28.4
ml-dtypes==0.2.0
monotonic==1.6
more-itertools==9.1.0
mpmath==1.3.0
msg-parser==1.2.0
msgpack==1.0.5
multidict==6.0.4
multiprocess==0.70.15
mypy-extensions==1.0.0
nest-asyncio==1.5.8
networkx==3.1
newspaper3k==0.2.8
nltk==3.8.1
nodeenv==1.7.0
numexpr==2.8.4
numpy==1.26.1
oauthlib==3.2.2
olefile==0.46
omegaconf==2.3.0
onnxruntime==1.14.1
openai==1.3.5
openapi-schema-pydantic==1.2.4
opencv-python==4.7.0.72
openpyxl==3.1.2
opt-einsum==3.3.0
ordered-set==4.1.0
outcome==1.2.0
overrides==7.4.0
packaging==23.2
pandas==1.5.3
pathos==0.3.1
pdf2image==1.16.3
pdfminer.six==20221105
pdfplumber==0.9.0
pexpect==4.8.0
Pillow==9.5.0
pinecone-client==2.2.4
pkginfo==1.9.6
platformdirs==2.6.2
Plim==1.0.0
pluggy==1.3.0
poetry==1.4.2
poetry-core==1.5.2
poetry-plugin-export==1.3.1
poppler-utils==0.1.0
portalocker==2.7.0
posthog==3.0.1
pox==0.3.3
ppft==1.7.6.7
pre-commit==3.2.2
proto-plus==1.22.3
protobuf==4.24.4
ptyprocess==0.7.0
pulsar-client==3.3.0
py==1.11.0
pyarrow==12.0.0
pyasn1==0.5.0
pyasn1-modules==0.3.0
pycocotools==2.0.6
pycparser==2.21
pydantic==2.4.2
pydantic_core==2.10.1
PyExecJS==1.5.1
pyfb==0.6.0
Pygments==2.15.1
PyJWT==2.7.0
pylance==0.8.7
PyMuPDF==1.22.3
pypandoc==1.11
pyparsing==3.0.9
pypdf==3.8.1
PyPDF2==3.0.1
PyPika==0.48.9
pyproject_hooks==1.0.0
pyrsistent==0.19.3
pyScss==1.4.0
PySocks==1.7.1
pytesseract==0.3.10
pytest==7.4.4
python-dateutil==2.8.2
python-docx==0.8.11
python-dotenv==1.0.0
python-Levenshtein==0.23.0
python-magic==0.4.27
python-multipart==0.0.6
python-pptx==0.6.21
pytz==2023.3
PyYAML==6.0
qdrant-client==1.6.4
rank-bm25==0.2.2
rapidfuzz==3.4.0
ratelimiter==1.2.0.post0
readability-lxml==0.8.1
redis==5.0.1
regex==2023.3.23
requests==2.31.0
requests-file==1.5.1
requests-oauthlib==1.3.1
requests-toolbelt==0.10.1
responses==0.18.0
retry==0.9.2
rfc3986==1.5.0
rich==13.0.1
rsa==4.9
s3transfer==0.7.0
safetensors==0.3.1
scikit-learn==1.2.2
scipy==1.10.1
selenium==4.9.1
semver==3.0.2
sentence-transformers==2.2.2
sentencepiece==0.1.98
sgmllib3k==1.0.0
shapely==2.0.2
shellingham==1.5.0.post1
simplejson==3.19.1
six @ file:///AppleInternal/Library/BuildRoots/9dd5efe2-7fad-11ee-b588-aa530c46a9ea/Library/Caches/com.apple.xbs/Sources/python3/six-1.15.0-py2.py3-none-any.whl
snakeviz==2.2.0
sniffio==1.3.0
sortedcontainers==2.4.0
soupsieve==2.5
SQLAlchemy==2.0.16
sqlean.py==0.21.8.4
starlette==0.27.0
stylus==0.1.2
sympy==1.11.1
tenacity==8.2.2
tensorboard==2.14.1
tensorboard-data-server==0.7.2
tensorflow==2.14.0
tensorflow-estimator==2.14.0
tensorflow-io-gcs-filesystem==0.34.0
tensorflow-macos==2.14.0
termcolor==2.3.0
threadpoolctl==3.1.0
tiktoken==0.4.0
timm==0.9.1
tinysegmenter==0.3
tldextract==5.1.1
tokenizers==0.13.3
tomli==2.0.1
tomlkit==0.11.8
torch==2.1.0
torchvision==0.15.1
tornado==6.2
tqdm==4.65.0
transformers==4.28.1
trio==0.22.0
trio-websocket==0.10.3
trove-classifiers==2023.5.2
typer==0.9.0
typing-inspect==0.8.0
typing_extensions==4.8.0
tzdata==2023.3
unstructured==0.6.6
unstructured-inference==0.4.4
urllib3==1.26.15
uvicorn==0.22.0
uvloop==0.17.0
virtualenv==20.21.1
Wand==0.6.11
watchfiles==0.19.0
webencodings==0.5.1
websockets==11.0.2
Werkzeug==2.3.6
wrapt==1.14.1
wsproto==1.2.0
xattr==0.10.1
XlsxWriter==3.1.0
xxhash==3.2.0
yarl==1.9.2
zipp==3.15.0
zstandard==0.21.0

Related Components

• LLMs/Chat Models
• Embedding Models
• Prompts / Prompt Templates / Prompt Selectors
• Output Parsers
• Document Loaders
• Vector Stores / Retrievers
• Memory
• Agents / Agent Executors
• Tools / Toolkits
• Chains
• Callbacks/Tracing
• Async

[dosubot]

🤖

Hey there, @JanHorcicka! Good to see you back in the LangChain repository. Let's dive into this.

The OutputParserException you're encountering when the response JSON contains a null query is likely due to the way the output parser in the LangChain library is handling the response. The output parser expects a ChatGeneration object and certain keys in the additional_kwargs of the message object and the function_call object. If these expectations are not met, an OutputParserException is raised.

Here's a brief explanation of the possible causes:

1. The output parser is expecting a ChatGeneration object, but it's receiving a different type. This is checked in the parse_result method of both OutputFunctionsParser and JsonOutputFunctionsParser classes. If the generation object is not an instance of ChatGeneration, an OutputParserException is raised.

2. The function_call key is not present in the additional_kwargs of the message object. This is checked in the parse_result method of both OutputFunctionsParser and JsonOutputFunctionsParser classes. If the function_call key is not found, an OutputParserException is raised.

3. The arguments key is not present in the function_call object or the JSON parsing of the arguments fails. This is checked in the parse_result method of the JsonOutputFunctionsParser class. If the arguments key is not found or the JSON parsing fails, an OutputParserException is raised.

To resolve this issue, ensure that the generation object is of type ChatGeneration, the function_call key is present in the additional_kwargs of the message object, and the arguments key is present in the function_call object and its value is a valid JSON string.

You can find more details in the source code of the LangChain library, specifically in the libs/langchain/langchain/output_parsers/openai_functions.py file.

As for the SelfQueryRetriever class, it is a retriever that uses a vector store and a language model to generate vector store queries. It is a subclass of the BaseRetriever class and provides functionalities for retrieving documents relevant to a query, initializing from a language model, and more. You can find more details in the libs/langchain/langchain/retrievers/self_query/base.py file.

I hope this helps! If you have any more questions or need further clarification, feel free to ask.

-Dosu

Sources

• libs/langchain/langchain/output_parsers/openai_functions.py
• libs/langchain/langchain/retrievers/self_query/base.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.