Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated reload_connections set false by default in in_opensearch.rb #140

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -605,10 +605,10 @@ request_timeout 15s # defaults to 5s

### reload_connections

You can tune how the opensearch-transport host reloading feature works. By default it will reload the host list from the server every 10,000th request to spread the load. This can be an issue if your OpenSearch cluster is behind a Reverse Proxy, as Fluentd process may not have direct network access to the OpenSearch nodes.
You can tune how the opensearch-transport host reloading feature works. By default it off but if you set true it will reload the host list from the server every 10,000th request to spread the load. This can be an issue if your OpenSearch cluster is behind a Reverse Proxy, as Fluentd process may not have direct network access to the OpenSearch nodes.

```
reload_connections false # defaults to true
reload_connections false # defaults to false
```

### reload_on_failure
Expand Down
10 changes: 7 additions & 3 deletions lib/fluent/plugin/in_opensearch.rb
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ class UnrecoverableRequestFailure < Fluent::UnrecoverableError; end
config_param :repeat, :bool, :default => true
config_param :http_backend, :enum, list: [:excon, :typhoeus], :default => :excon
config_param :request_timeout, :time, :default => 5
config_param :reload_connections, :bool, :default => true
config_param :reload_connections, :bool, :default => false
config_param :reload_on_failure, :bool, :default => false
config_param :resurrect_after, :time, :default => 60
config_param :reload_after, :integer, :default => DEFAULT_RELOAD_AFTER
Expand Down Expand Up @@ -215,7 +215,7 @@ def reachable_host?(host)
host: ["#{host[:scheme]}://#{host[:host]}:#{host[:port]}"],
user: host[:user],
password: host[:password],
reload_connections: @reload_connections,
# reload_connections: @reload_connections,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, could you describe why this line is commented out?
Would you mind if you revive this line?

request_timeout: @request_timeout,
resurrect_after: @resurrect_after,
reload_on_failure: @reload_on_failure,
Expand Down Expand Up @@ -343,14 +343,18 @@ def update_retry_state(error=nil)
@retry = retry_state(@retry_randomize)
end
@retry.step
if error.message.include?('EOFError (EOFError)')
log.error("Restart plugin because hit error #{error.message}")
exit(1)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exit(1) caused entire termination. This is brutal manner for handling this exception. :(

end
#Raise error if the retry limit has been reached
raise "Hit limit for retries. retry_times: #{@retry.steps}, error: #{error.message}" if @retry.limit?
#Retry if the limit hasn't been reached
log.warn("failed to connect or search.", retry_times: @retry.steps, next_retry_time: @retry.next_time.round, error: error.message)
sleep(@retry.next_time - Time.now)
else
unless @retry.nil?
log.debug("retry succeeded.")
log.info("retry succeeded.")
@retry = nil
end
end
Expand Down
Loading