Skip to content

(MODULES-1394) replace validate_db_connection type with custom type #879

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 12, 2017
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
135 changes: 63 additions & 72 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -283,16 +283,16 @@ Only the specified parameters are recognized in the template. The `recovery.conf

### Validate connectivity

To validate client connections to a remote PostgreSQL database before starting dependent tasks, use the `postgresql::validate_db_connection` resource. You can use this on any node where the PostgreSQL client software is installed. It is often chained to other tasks such as starting an application server or performing a database migration.
To validate client connections to a remote PostgreSQL database before starting dependent tasks, use the `postgresql_conn_validator` resource. You can use this on any node where the PostgreSQL client software is installed. It is often chained to other tasks such as starting an application server or performing a database migration.

Example usage:

```puppet
postgresql::validate_db_connection { 'validate my postgres connection':
database_host => 'my.postgres.host',
database_username => 'mydbuser',
database_password => 'mydbpassword',
database_name => 'mydbname',
postgresql_conn_validator { 'validate my postgres connection':
host => 'my.postgres.host',
db_username => 'mydbuser',
db_password => 'mydbpassword',
db_name => 'mydbname',
}->
exec { 'rake db:migrate':
cwd => '/opt/myrubyapp',
Expand Down Expand Up @@ -332,13 +332,13 @@ The postgresql module comes with many options for configuring the server. While
* [postgresql::server::schema](#postgresqlserverschema)
* [postgresql::server::table_grant](#postgresqlservertable_grant)
* [postgresql::server::tablespace](#postgresqlservertablespace)
* [postgresql::validate_db_connection](#postgresqlvalidate_db_connection)

**Types:**

* [postgresql_psql](#custom-resource-postgresql_psql)
* [postgresql_replication_slot](#custom-resource-postgresql_replication_slot)
* [postgresql_conf](#custom-resource-postgresql_conf)
* [postgresql_conn_validator](#custom-resource-postgresql_conn_validator)

**Functions:**

Expand Down Expand Up @@ -367,13 +367,6 @@ Sets the name of the PostgreSQL client package.

Default value: 'file'.

##### `validcon_script_path`

Specifies the path to validate the connection script.


Default value: '/usr/local/bin/validate_postgresql_connection.sh'.

#### postgresql::lib::docs

Installs PostgreSQL bindings for Postgres-Docs. Set the following parameters if you have a custom version you would like to install.
Expand Down Expand Up @@ -1543,64 +1536,6 @@ Specifies the name of the tablespace.

Default value: the namevar.

#### postgresql::validate_db_connection

Validates client connection with a remote PostgreSQL database.

##### `connect_settings`

Specifies a hash of environment variables used when connecting to a remote server. This is an alternative to providing individual parameters (`database_host`, etc). If provided, the individual parameters take precedence.

##### `create_db_first`

Ensures that the database is created before running the test. This only works if your test is local.

Default value: `true`.

##### `database_host`

Sets the hostname of the database you wish to test.

Default value: `undef`, which generally uses the designated local Unix socket.

##### `database_name`

Specifies the name of the database you wish to test.

Default value: 'postgres'.

##### `database_port`

Defines the port to use when connecting.

Default value: `undef`, which generally defaults to port 5432 depending on your PostgreSQL packaging.

##### `database_password`

Specifies the password to connect with. Can be left blank, not recommended.

##### `database_username`

Specifies the username to connect with.

Default value: `undef`.

When using a Unix socket and ident auth, this is the user you are running as.

**If the host is remote you must provide a username.**

##### `run_as`

Specifies the user to run the `psql` command as. This is important when trying to connect to a database locally using Unix sockets and `ident` authentication. Not needed for remote testing.

##### `sleep`

Sets the number of seconds to sleep for before trying again after a failure.

##### `tries`

Sets the number of attempts after failure before giving up and failing the resource.

### Types

#### postgresql_psql
Expand Down Expand Up @@ -1703,6 +1638,62 @@ Specifies the name of the slot to create. Must be a valid replication slot name.

This is the namevar.

#### postgresql_conn_validator

Validate the connection to a local or remote PostgreSQL database using this type.

##### `connect_settings`

Specifies a hash of environment variables used when connecting to a remote server. This is an alternative to providing individual parameters (`host`, etc). If provided, the individual parameters take precedence.

Default value: {}

##### `db_name`

Specifies the name of the database you wish to test.

Default value: ''

##### `db_password`

Specifies the password to connect with. Can be left blank if `.pgpass` is being used, otherwise not recommended.

Default value: ''

##### `db_username`

Specifies the username to connect with.

Default value: ''

When using a Unix socket and ident auth, this is the user you are running as.

##### `host`

Sets the hostname of the database you wish to test.

Default value: '', which generally uses the designated local Unix socket.

**If the host is remote you must provide a username.**

##### `port`

Defines the port to use when connecting.

Default value: ''

##### `run_as`

Specifies the user to run the `psql` command as. This is important when trying to connect to a database locally using Unix sockets and `ident` authentication. Not needed for remote testing.

##### `sleep`

Sets the number of seconds to sleep for before trying again after a failure.

##### `tries`

Sets the number of attempts after failure before giving up and failing the resource.

### Functions

#### postgresql_password
Expand Down
43 changes: 43 additions & 0 deletions lib/puppet/provider/postgresql_conn_validator/ruby.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
$LOAD_PATH.unshift(File.join(File.dirname(__FILE__),"..","..",".."))
require 'puppet/util/postgresql_validator'

# This file contains a provider for the resource type `postgresql_conn_validator`,
# which validates the puppetdb connection by attempting an https connection.

Puppet::Type.type(:postgresql_conn_validator).provide(:ruby) do
desc "A provider for the resource type `postgresql_conn_validator`,
which validates the PostgreSQL connection by attempting a query
to the target PostgreSQL server."

# Test to see if the resource exists, returns true if it does, false if it
# does not.
#
# Here we simply monopolize the resource API, to execute a test to see if the
# database is connectable. When we return a state of `false` it triggers the
# create method where we can return an error message.
#
# @return [bool] did the test succeed?
def exists?
validator.attempt_connection(resource[:sleep], resource[:tries])
end

# This method is called when the exists? method returns false.
#
# @return [void]
def create
# If `#create` is called, that means that `#exists?` returned false, which
# means that the connection could not be established... so we need to
# cause a failure here.
raise Puppet::Error, "Unable to connect to PostgreSQL server! (#{resource[:host]}:#{resource[:port]})"
end

# Returns the existing validator, if one exists otherwise creates a new object
# from the class.
#
# @api private
def validator
@validator ||= Puppet::Util::PostgresqlValidator.new(resource)
end

end

82 changes: 82 additions & 0 deletions lib/puppet/type/postgresql_conn_validator.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
Puppet::Type.newtype(:postgresql_conn_validator) do

@doc = "Verify that a connection can be successfully established between a node
and the PostgreSQL server. Its primary use is as a precondition to
prevent configuration changes from being applied if the PostgreSQL
server cannot be reached, but it could potentially be used for other
purposes such as monitoring."

ensurable do
defaultvalues
defaultto :present
end

newparam(:name, :namevar => true) do
desc 'An arbitrary name used as the identity of the resource.'
end

newparam(:db_name) do
desc "The name of the database you are trying to validate a connection with."
end

newparam(:db_username) do
desc "A user that has access to the target PostgreSQL database."
end

newparam(:db_password) do
desc "The password required to access the target PostgreSQL database."
end

newparam(:host) do
desc 'The DNS name or IP address of the server where PostgreSQL should be running.'
end

newparam(:port) do
desc 'The port that the PostgreSQL server should be listening on.'

validate do |value|
if value
value =~ /[0-9]+/
end
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it is better to check this as an integer.

end
end

newparam(:connect_settings) do
desc 'Hash of environment variables for connection to a db.'

end

newparam(:sleep) do
desc "The length of sleep time between connection tries."

validate do |value|
Integer(value)
end
munge do |value|
Integer(value)
end

defaultto 2
end

newparam(:tries) do
desc "The number of tries to validate the connection to the target PostgreSQL database."

validate do |value|
Integer(value)
end
munge do |value|
Integer(value)
end

defaultto 10
end

newparam(:psql_path) do
desc "Path to the psql command."
end

newparam(:run_as) do
desc "System user that will run the psql command."
end
end
64 changes: 64 additions & 0 deletions lib/puppet/util/postgresql_validator.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
module Puppet
module Util
class PostgresqlValidator
attr_reader :resource

def initialize(resource)
@resource = resource
end

def build_psql_cmd
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Postgres has the PQping() command on its binary protocol to issue a connection check. It can be used with pg_isready binary. I think it would make a good default provider. It would be more efficient and easier to use, because you don't need to pass authentication. It also supports a custom timeout.

final_cmd = []

cmd_init = "#{@resource[:psql_path]} --tuples-only --quiet "

final_cmd.push cmd_init

cmd_parts = {
:host => "-h #{@resource[:host]}",
:port => "-p #{@resource[:port]}",
:db_username => "-U #{@resource[:db_username]}",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some short and some long arguments don't please my eyes. I prefer --xxx=yyy format on scripts when possible to avoid argument parsing issues.

:db_name => "--dbname #{@resource[:db_name]}"
}

cmd_parts[:db_password] = "-w " if @resource[:db_password]

cmd_parts.each do |k,v|
final_cmd.push v if @resource[k]
end

final_cmd.join ' '
end

def parse_connect_settings
c_settings = @resource[:connect_settings] || {}
c_settings.merge! ({ 'PGPASSWORD' => @resource[:db_password] }) if @resource[:db_password]
return c_settings.map { |k,v| "#{k}=#{v}" }
end

def attempt_connection(sleep_length, tries)
(0..tries-1).each do |try|
Puppet.debug "PostgresqlValidator.attempt_connection: Attempting connection to #{@resource[:db_name]}"
if execute_command =~ /1/
Puppet.debug "PostgresqlValidator.attempt_connection: Connection to #{@resource[:db_name]} successful!"
return true
else
Puppet.warning "PostgresqlValidator.attempt_connection: Sleeping for #{sleep_length} seconds"
sleep sleep_length
end
end
false
end

private

def execute_command
Execution.execute(build_validate_cmd, :uid => @resource[:run_as])
end

def build_validate_cmd
"/bin/echo 'SELECT 1' | #{parse_connect_settings.join(' ')} #{build_psql_cmd} "
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't spawn another process but use --command='SELECT 1'.

end
end
end
end
2 changes: 1 addition & 1 deletion manifests/server/db.pp
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
privilege => $grant,
db => $dbname,
role => $user,
} -> Postgresql::Validate_db_connection<| database_name == $dbname |>
} -> Postgresql_conn_validator<| db_name == $dbname |>
}

if($tablespace != undef and defined(Postgresql::Server::Tablespace[$tablespace])) {
Expand Down
Loading