rpms/389-ds-base
1
0
Fork 0
mirror of https://git.centos.org/rpms/389-ds-base.git synced 2025-02-23 16:22:54 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

import 389-ds-base-1.3.10.2-1.el7

Browse source
This commit is contained in:
CentOS Sources 2020-05-20 11:15:39 -04:00
parent 458e05c72c
commit dbe35a69b2
11 changed files with 48 additions and 1399 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
.389-ds-base.metadata
View file

@ -1 +1 @@
c995af6f5693f698c29b72ebfdbc0e60a72cc517 SOURCES/389-ds-base-1.3.10.1.tar.bz2
5ea563775de60788f87373327d90c09ce37a574b SOURCES/389-ds-base-1.3.10.2.tar.bz2

2
.gitignore vendored
View file

@ -1 +1 @@
SOURCES/389-ds-base-1.3.10.1.tar.bz2
SOURCES/389-ds-base-1.3.10.2.tar.bz2

53
SOURCES/0000-CVE-2019-14824-BZ-1748199-deref-plugin-displays-rest.patch
View file

@ -1,53 +0,0 @@
From 63fa3ee665b66b36321489c090b24811838837c0 Mon Sep 17 00:00:00 2001
From: Mark Reynolds <mreynolds@redhat.com>
Date: Tue, 3 Sep 2019 13:15:45 -0400
Subject: [PATCH] CVE-2019-14824 (BZ#1748199) - deref plugin displays
restricted attributes
Bug Description: If there is an ACI that allows "search" access to an attribute,
the deref plugin access control checks sees this is a "read"
privilege and returns the attribute's value.
Fix description: For deref plugin we are only concerned with "read" access, not
"search" access. Removed the SLAPI_ACL_SEARCH right flag when
checking access for an attribute.
Reviewed by: lkrispen & tbordaz(Thanks!)
---
ldap/servers/plugins/deref/deref.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/ldap/servers/plugins/deref/deref.c b/ldap/servers/plugins/deref/deref.c
index cb5ebb830..ec1884ba3 100644
--- a/ldap/servers/plugins/deref/deref.c
+++ b/ldap/servers/plugins/deref/deref.c
@@ -573,7 +573,7 @@ deref_do_deref_attr(Slapi_PBlock *pb, BerElement *ctrlber, const char *derefdn,
Slapi_Entry **entries = NULL;
int rc;
- /* If the access check on the attributes is done without retrieveing the entry
+ /* If the access check on the attributes is done without retrieving the entry
* it cannot handle acis which need teh entry, eg to apply a targetfilter rule
* So the determination of attrs which can be dereferenced is delayed
*/
@@ -596,7 +596,7 @@ deref_do_deref_attr(Slapi_PBlock *pb, BerElement *ctrlber, const char *derefdn,
int ii;
int needattrvals = 1; /* need attrvals sequence? */
if (deref_check_access(pb, entries[0], derefdn, attrs, &retattrs,
- (SLAPI_ACL_SEARCH | SLAPI_ACL_READ))) {
+ SLAPI_ACL_READ)) {
slapi_log_err(SLAPI_LOG_PLUGIN, DEREF_PLUGIN_SUBSYSTEM,
"deref_do_deref_attr - The client does not have permission to read the requested "
"attributes in entry %s\n",
@@ -714,7 +714,7 @@ deref_pre_entry(Slapi_PBlock *pb)
attrs[1] = NULL;
if (deref_check_access(pb, ent, NULL, attrs, &retattrs,
- (SLAPI_ACL_SEARCH | SLAPI_ACL_READ))) {
+ SLAPI_ACL_READ)) {
slapi_log_err(SLAPI_LOG_PLUGIN, DEREF_PLUGIN_SUBSYSTEM,
"deref_pre_entry - The client does not have permission to read attribute %s in entry %s\n",
spec->derefattr, slapi_entry_get_dn_const(ent));
--
2.21.0

250
SOURCES/0001-Issue-50525-nsslapd-defaultnamingcontext-does-not-ch.patch
View file

@ -1,250 +0,0 @@
From d33743c8604ff4f97947dad14fddab0691e3d19e Mon Sep 17 00:00:00 2001
From: Mark Reynolds <mreynolds@redhat.com>
Date: Thu, 1 Aug 2019 16:50:34 -0400
Subject: [PATCH] Issue 50525 - nsslapd-defaultnamingcontext does not change
when the assigned suffix gets deleted
Bug Description:
If you delete the suffix that is set as the default naming context, the attribute
is not reset.
Also using dsconf to delete a backend/suffix fails if there are vlv indexes, encrypted
attributes, or replication is configured.
Fix Description:
As for the default naming context, if there is a second suffix configured, it will be
automatically set as the new default naming context, otherwise the attribute is not
modified.
For dsconf backend delete issue, it now checks and removes replication configuration
and agreements, and removes all the child entries under the backend entry.
relates: https://pagure.io/389-ds-base/issue/50525
Reviewed by: spichugi(Thanks!)
---
.../be_del_and_default_naming_attr_test.py | 90 +++++++++++++++++++
ldap/servers/slapd/mapping_tree.c | 50 ++++++-----
src/lib389/lib389/backend.py | 17 ++--
src/lib389/lib389/replica.py | 2 +-
4 files changed, 132 insertions(+), 27 deletions(-)
create mode 100644 dirsrvtests/tests/suites/mapping_tree/be_del_and_default_naming_attr_test.py
diff --git a/dirsrvtests/tests/suites/mapping_tree/be_del_and_default_naming_attr_test.py b/dirsrvtests/tests/suites/mapping_tree/be_del_and_default_naming_attr_test.py
new file mode 100644
index 000000000..34a2de2ad
--- /dev/null
+++ b/dirsrvtests/tests/suites/mapping_tree/be_del_and_default_naming_attr_test.py
@@ -0,0 +1,90 @@
+import logging
+import pytest
+import os
+from lib389._constants import DEFAULT_SUFFIX
+from lib389.topologies import topology_m1 as topo
+from lib389.backend import Backends
+from lib389.encrypted_attributes import EncryptedAttrs
+
+DEBUGGING = os.getenv("DEBUGGING", default=False)
+if DEBUGGING:
+ logging.getLogger(__name__).setLevel(logging.DEBUG)
+else:
+ logging.getLogger(__name__).setLevel(logging.INFO)
+log = logging.getLogger(__name__)
+
+SECOND_SUFFIX = 'o=namingcontext'
+THIRD_SUFFIX = 'o=namingcontext2'
+
+def test_be_delete(topo):
+ """Test that we can delete a backend that contains replication
+ configuration and encrypted attributes. The default naming
+ context should also be updated to reflect the next available suffix
+
+ :id: 5208f897-7c95-4925-bad0-9ceb95fee678
+ :setup: Master Instance
+ :steps:
+ 1. Create second backend/suffix
+ 2. Add an encrypted attribute to the default suffix
+ 2. Delete default suffix
+ 3. Check the nsslapd-defaultnamingcontext is updated
+ 4. Delete the last backend
+ 5. Check the namingcontext has not changed
+ 6. Add new backend
+ 7. Set default naming context
+ 8. Verify the naming context is correct
+ :expectedresults:
+ 1. Success
+ 2. Success
+ 3. Success
+ 4. Success
+ 5. Success
+ 6. Success
+ 7. Success
+ 8. Success
+ """
+
+ inst = topo.ms["master1"]
+
+ # Create second suffix
+ backends = Backends(inst)
+ default_backend = backends.get(DEFAULT_SUFFIX)
+ new_backend = backends.create(properties={'nsslapd-suffix': SECOND_SUFFIX,
+ 'name': 'namingRoot'})
+
+ # Add encrypted attribute entry under default suffix
+ encrypt_attrs = EncryptedAttrs(inst, basedn='cn=encrypted attributes,{}'.format(default_backend.dn))
+ encrypt_attrs.create(properties={'cn': 'employeeNumber', 'nsEncryptionAlgorithm': 'AES'})
+
+ # Delete default suffix
+ default_backend.delete()
+
+ # Check that the default naming context is set to the new/second suffix
+ default_naming_ctx = inst.config.get_attr_val_utf8('nsslapd-defaultnamingcontext')
+ assert default_naming_ctx == SECOND_SUFFIX
+
+ # delete new backend, but the naming context should not change
+ new_backend.delete()
+
+ # Check that the default naming context is still set to the new/second suffix
+ default_naming_ctx = inst.config.get_attr_val_utf8('nsslapd-defaultnamingcontext')
+ assert default_naming_ctx == SECOND_SUFFIX
+
+ # Add new backend
+ new_backend = backends.create(properties={'nsslapd-suffix': THIRD_SUFFIX,
+ 'name': 'namingRoot2'})
+
+ # manaully set naming context
+ inst.config.set('nsslapd-defaultnamingcontext', THIRD_SUFFIX)
+
+ # Verify naming context is correct
+ default_naming_ctx = inst.config.get_attr_val_utf8('nsslapd-defaultnamingcontext')
+ assert default_naming_ctx == THIRD_SUFFIX
+
+
+if __name__ == '__main__':
+ # Run isolated
+ # -s for DEBUG mode
+ CURRENT_FILE = os.path.realpath(__file__)
+ pytest.main(["-s", CURRENT_FILE])
+
diff --git a/ldap/servers/slapd/mapping_tree.c b/ldap/servers/slapd/mapping_tree.c
index 834949a67..25e9fb80c 100644
--- a/ldap/servers/slapd/mapping_tree.c
+++ b/ldap/servers/slapd/mapping_tree.c
@@ -1521,26 +1521,36 @@ done:
strcpy_unescape_value(escaped, suffix);
}
if (escaped && (0 == strcasecmp(escaped, default_naming_context))) {
- int rc = _mtn_update_config_param(LDAP_MOD_DELETE,
- CONFIG_DEFAULT_NAMING_CONTEXT,
- NULL);
- if (rc) {
- slapi_log_err(SLAPI_LOG_ERR,
- "mapping_tree_entry_delete_callback",
- "deleting config param %s failed: RC=%d\n",
- CONFIG_DEFAULT_NAMING_CONTEXT, rc);
- }
- if (LDAP_SUCCESS == rc) {
- char errorbuf[SLAPI_DSE_RETURNTEXT_SIZE] = {0};
- /* Removing defaultNamingContext from cn=config entry
- * was successful. The remove does not reset the
- * global parameter. We need to reset it separately. */
- if (config_set_default_naming_context(
- CONFIG_DEFAULT_NAMING_CONTEXT,
- NULL, errorbuf, CONFIG_APPLY)) {
- slapi_log_err(SLAPI_LOG_ERR, "mapping_tree_entry_delete_callback",
- "Setting NULL to %s failed. %s\n",
- CONFIG_DEFAULT_NAMING_CONTEXT, errorbuf);
+ /*
+ * We can not delete the default naming attribute, so instead
+ * replace it only if there is another suffix available
+ */
+ void *node = NULL;
+ Slapi_DN *sdn;
+ sdn = slapi_get_first_suffix(&node, 0);
+ if (sdn) {
+ char *replacement_suffix = (char *)slapi_sdn_get_dn(sdn);
+ int rc = _mtn_update_config_param(LDAP_MOD_REPLACE,
+ CONFIG_DEFAULT_NAMING_CONTEXT,
+ replacement_suffix);
+ if (rc) {
+ slapi_log_err(SLAPI_LOG_ERR,
+ "mapping_tree_entry_delete_callback",
+ "replacing config param %s failed: RC=%d\n",
+ CONFIG_DEFAULT_NAMING_CONTEXT, rc);
+ }
+ if (LDAP_SUCCESS == rc) {
+ char errorbuf[SLAPI_DSE_RETURNTEXT_SIZE] = {0};
+ /* Replacing defaultNamingContext from cn=config entry
+ * was successful. The replace does not reset the
+ * global parameter. We need to reset it separately. */
+ if (config_set_default_naming_context(
+ CONFIG_DEFAULT_NAMING_CONTEXT,
+ replacement_suffix, errorbuf, CONFIG_APPLY)) {
+ slapi_log_err(SLAPI_LOG_ERR, "mapping_tree_entry_delete_callback",
+ "Setting %s tp %s failed. %s\n",
+ CONFIG_DEFAULT_NAMING_CONTEXT, replacement_suffix, errorbuf);
+ }
}
}
}
diff --git a/src/lib389/lib389/backend.py b/src/lib389/lib389/backend.py
index 6f4c8694e..4d32038f6 100644
--- a/src/lib389/lib389/backend.py
+++ b/src/lib389/lib389/backend.py
@@ -17,6 +17,7 @@ from lib389 import Entry
from lib389._mapped_object import DSLdapObjects, DSLdapObject
from lib389.mappingTree import MappingTrees, MappingTree
from lib389.exceptions import NoSuchEntryError, InvalidArgumentError
+from lib389.replica import Replicas
# We need to be a factor to the backend monitor
from lib389.monitor import MonitorBackend
@@ -507,20 +508,24 @@ class Backend(DSLdapObject):
mt = self._mts.get(selector=bename)
# Assert the type is "backend"
# Are these the right types....?
- if mt.get_attr_val('nsslapd-state') != ensure_bytes('backend'):
+ if mt.get_attr_val('nsslapd-state').lower() != ensure_bytes('backend'):
raise ldap.UNWILLING_TO_PERFORM('Can not delete the mapping tree, not for a backend! You may need to delete this backend via cn=config .... ;_; ')
+
+ # Delete replicas first
+ try:
+ Replicas(self._instance).get(mt.get_attr_val_utf8('cn')).delete()
+ except ldap.NO_SUCH_OBJECT:
+ # No replica, no problem
+ pass
+
# Delete our mapping tree if it exists.
mt.delete()
except ldap.NO_SUCH_OBJECT:
# Righto, it's already gone! Do nothing ...
pass
- # Delete all our related indices
- self._instance.index.delete_all(bename)
# Now remove our children, this is all ldbm config
- self._instance.delete_branch_s(self._dn, ldap.SCOPE_ONELEVEL)
- # The super will actually delete ourselves.
- super(Backend, self).delete()
+ self._instance.delete_branch_s(self._dn, ldap.SCOPE_SUBTREE)
def _lint_mappingtree(self):
"""Backend lint
diff --git a/src/lib389/lib389/replica.py b/src/lib389/lib389/replica.py
index cdd0a9729..7b45683d9 100644
--- a/src/lib389/lib389/replica.py
+++ b/src/lib389/lib389/replica.py
@@ -458,7 +458,7 @@ class ReplicaLegacy(object):
try:
self.deleteAgreements(nsuffix)
except ldap.LDAPError as e:
- self.log.fatal('Failed to delete replica agreements!')
+ self.log.fatal('Failed to delete replica agreements! ' + str(e))
raise
# Delete the replica
--
2.21.0

146
SOURCES/0002-Issue-50530-Directory-Server-not-RFC-4511-compliant-.patch
View file

@ -1,146 +0,0 @@
From 51ea1d34b861dfffb12fbe6be4e23d9342fd0fe2 Mon Sep 17 00:00:00 2001
From: Mark Reynolds <mreynolds@redhat.com>
Date: Fri, 2 Aug 2019 14:36:24 -0400
Subject: [PATCH] Issue 50530 - Directory Server not RFC 4511 compliant with
requested attr "1.1"
Bug Description: A regression was introduced some time back that changed the
behavior of how the server handled the "1.1" requested attribute
in a search request. If "1.1" was requested along with other
attributes then no attibutes were returned, but in this case "1.1"
is expected to be ignroed.
Fix Description: Only comply with "1.1" if it is the only requested attribute
relates: https://pagure.io/389-ds-base/issue/50530
Reviewed by: firstyear(Thanks!)
---
dirsrvtests/tests/suites/basic/basic_test.py | 57 +++++++++++++++++---
ldap/servers/slapd/result.c | 7 ++-
2 files changed, 57 insertions(+), 7 deletions(-)
diff --git a/dirsrvtests/tests/suites/basic/basic_test.py b/dirsrvtests/tests/suites/basic/basic_test.py
index 0f7536b63..cea4f6bfe 100644
--- a/dirsrvtests/tests/suites/basic/basic_test.py
+++ b/dirsrvtests/tests/suites/basic/basic_test.py
@@ -28,6 +28,7 @@ log = logging.getLogger(__name__)
USER1_DN = 'uid=user1,' + DEFAULT_SUFFIX
USER2_DN = 'uid=user2,' + DEFAULT_SUFFIX
USER3_DN = 'uid=user3,' + DEFAULT_SUFFIX
+USER4_DN = 'uid=user4,' + DEFAULT_SUFFIX
ROOTDSE_DEF_ATTR_LIST = ('namingContexts',
'supportedLDAPVersion',
@@ -409,8 +410,8 @@ def test_basic_acl(topology_st, import_example_ldif):
'uid': 'user1',
'userpassword': PASSWORD})))
except ldap.LDAPError as e:
- log.fatal('test_basic_acl: Failed to add test user ' + USER1_DN
- + ': error ' + e.message['desc'])
+ log.fatal('test_basic_acl: Failed to add test user ' + USER1_DN +
+ ': error ' + e.message['desc'])
assert False
try:
@@ -421,8 +422,8 @@ def test_basic_acl(topology_st, import_example_ldif):
'uid': 'user2',
'userpassword': PASSWORD})))
except ldap.LDAPError as e:
- log.fatal('test_basic_acl: Failed to add test user ' + USER1_DN
- + ': error ' + e.message['desc'])
+ log.fatal('test_basic_acl: Failed to add test user ' + USER1_DN +
+ ': error ' + e.message['desc'])
assert False
#
@@ -572,6 +573,50 @@ def test_basic_searches(topology_st, import_example_ldif):
log.info('test_basic_searches: PASSED')
+@pytest.fixture(scope="module")
+def add_test_entry(topology_st, request):
+ # Add test entry
+ topology_st.standalone.add_s(Entry((USER4_DN,
+ {'objectclass': "top extensibleObject".split(),
+ 'cn': 'user1', 'uid': 'user1'})))
+
+
+search_params = [(['1.1'], 'cn', False),
+ (['1.1', 'cn'], 'cn', True),
+ (['+'], 'nsUniqueId', True),
+ (['*'], 'cn', True),
+ (['cn'], 'cn', True)]
+@pytest.mark.parametrize("attrs, attr, present", search_params)
+def test_search_req_attrs(topology_st, add_test_entry, attrs, attr, present):
+ """Test requested attributes in search operations.
+ :id: 426a59ff-49b8-4a70-b377-0c0634a29b6e
+ :setup: Standalone instance
+ :steps:
+ 1. Test "1.1" does not return any attributes.
+ 2. Test "1.1" is ignored if there are other requested attributes
+ 3. Test "+" returns all operational attributes
+ 4. Test "*" returns all attributes
+ 5. Test requested attributes
+
+ :expectedresults:
+ 1. Success
+ 2. Success
+ 3. Success
+ 4. Success
+ 5. Success
+ """
+
+ log.info("Testing attrs: {} attr: {} present: {}".format(attrs, attr, present))
+ entry = topology_st.standalone.search_s(USER4_DN,
+ ldap.SCOPE_BASE,
+ 'objectclass=top',
+ attrs)
+ if present:
+ assert entry[0].hasAttr(attr)
+ else:
+ assert not entry[0].hasAttr(attr)
+
+
def test_basic_referrals(topology_st, import_example_ldif):
"""Test LDAP server in referral mode.
@@ -716,8 +761,8 @@ def test_basic_systemctl(topology_st, import_example_ldif):
log.info('Attempting to start the server with broken dse.ldif...')
try:
topology_st.standalone.start()
- except:
- log.info('Server failed to start as expected')
+ except Exception as e:
+ log.info('Server failed to start as expected: ' + str(e))
log.info('Check the status...')
assert (not topology_st.standalone.status())
log.info('Server failed to start as expected')
diff --git a/ldap/servers/slapd/result.c b/ldap/servers/slapd/result.c
index d9f431cc5..34ddd8566 100644
--- a/ldap/servers/slapd/result.c
+++ b/ldap/servers/slapd/result.c
@@ -1546,6 +1546,8 @@ send_ldap_search_entry_ext(
* "+" means all operational attributes (rfc3673)
* operational attributes are only retrieved if they are named
* specifically or when "+" is specified.
+ * In the case of "1.1", if there are other requested attributes
+ * then "1.1" should be ignored.
*/
/* figure out if we want all user attributes or no attributes at all */
@@ -1560,7 +1562,10 @@ send_ldap_search_entry_ext(
if (strcmp(LDAP_ALL_USER_ATTRS, attrs[i]) == 0) {
alluserattrs = 1;
} else if (strcmp(LDAP_NO_ATTRS, attrs[i]) == 0) {
- noattrs = 1;
+ /* "1.1" is only valid if it's the only requested attribute */
+ if (i == 0 && attrs[1] == NULL) {
+ noattrs = 1;
+ }
} else if (strcmp(LDAP_ALL_OPERATIONAL_ATTRS, attrs[i]) == 0) {
alloperationalattrs = 1;
} else {
--
2.21.0

37
SOURCES/0003-Issue-50529-LDAP-server-returning-PWP-controls-in-di.patch
View file

@ -1,37 +0,0 @@
From 606b7b6a45f6e2014119d0716774323f30862e0c Mon Sep 17 00:00:00 2001
From: Mark Reynolds <mreynolds@redhat.com>
Date: Fri, 2 Aug 2019 12:07:07 -0400
Subject: [PATCH] Issue 50529 - LDAP server returning PWP controls in
different sequence
Description: The server returns password policy controls in different orders
depending on the state of grace logins. The requested control,
if any, should be returned first, followed by any controls the
server might add.
relates: https://pagure.io/389-ds-base/issue/50529
Reviewed by: mreynolds (one line commit rule)
---
ldap/servers/slapd/pw_mgmt.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/ldap/servers/slapd/pw_mgmt.c b/ldap/servers/slapd/pw_mgmt.c
index befac50cd..ca76fc12f 100644
--- a/ldap/servers/slapd/pw_mgmt.c
+++ b/ldap/servers/slapd/pw_mgmt.c
@@ -207,10 +207,10 @@ skip:
/* password expired and user exceeded limit of grace attemps.
* Send result and also the control */
- slapi_add_pwd_control(pb, LDAP_CONTROL_PWEXPIRED, 0);
if (pwresponse_req) {
slapi_pwpolicy_make_response_control(pb, -1, -1, LDAP_PWPOLICY_PWDEXPIRED);
}
+ slapi_add_pwd_control(pb, LDAP_CONTROL_PWEXPIRED, 0);
slapi_send_ldap_result(pb, LDAP_INVALID_CREDENTIALS, NULL,
"password expired!", 0, NULL);
--
2.21.0

682
SOURCES/0004-Issue-50538-cleanAllRUV-task-limit-is-not-enforced-f.patch
View file

@ -1,682 +0,0 @@
From 59f03e332061b2c68bb53eed5949ddcfdc563300 Mon Sep 17 00:00:00 2001
From: Mark Reynolds <mreynolds@redhat.com>
Date: Wed, 7 Aug 2019 20:36:53 -0400
Subject: [PATCH] Issue 50538 - cleanAllRUV task limit is not enforced for
replicated tasks
Bug Description:
There is a hard limit of 64 concurrent cleanAllRUV tasks, but this limit is
only enforced when creating "new" tasks. It was not enforced when a task was
received via an extended operation. There were also race conditions in the
existing logic that allowed the array of cleaned rids to get corrupted . This
allowed for a very large number of task threads to be created.
Fix Description:
Maintain a new counter to keep track of the number of clean and abort threads
to make sure it never over runs the rid array buffers.
relates: https://pagure.io/389-ds-base/issue/50538
Reviewed by: lkrispenz(Thanks!)
---
.../suites/replication/cleanallruv_test.py | 47 +++-
ldap/servers/plugins/replication/repl5.h | 7 +-
.../replication/repl5_replica_config.c | 247 ++++++++++--------
ldap/servers/plugins/replication/repl_extop.c | 19 +-
4 files changed, 202 insertions(+), 118 deletions(-)
diff --git a/dirsrvtests/tests/suites/replication/cleanallruv_test.py b/dirsrvtests/tests/suites/replication/cleanallruv_test.py
index 620a53e1a..43801dd52 100644
--- a/dirsrvtests/tests/suites/replication/cleanallruv_test.py
+++ b/dirsrvtests/tests/suites/replication/cleanallruv_test.py
@@ -1,5 +1,5 @@
# --- BEGIN COPYRIGHT BLOCK ---
-# Copyright (C) 2016 Red Hat, Inc.
+# Copyright (C) 2019 Red Hat, Inc.
# All rights reserved.
#
# License: GPL (version 3 or any later version).
@@ -7,7 +7,6 @@
# --- END COPYRIGHT BLOCK ---
#
import threading
-
import pytest
from lib389.tasks import *
from lib389.utils import *
@@ -859,6 +858,50 @@ def test_multiple_tasks_with_force(topology_m4):
restore_master4(topology_m4)
+def test_max_tasks(topology_m4):
+ """Test we can not create more than 64 cleaning tasks
+
+ :id: c34d0b40-3c3e-4f53-8656-5e4c2a310a1f
+ :setup: Replication setup with four masters
+ :steps:
+ 1. Stop masters 3 & 4
+ 2. Create over 64 tasks between m1 and m2
+ 3. Check logs to see if (>65) tasks were rejected
+
+ :expectedresults:
+ 1. Success
+ 2. Success
+ 3. Success
+ """
+
+ # Stop masters 3 & 4
+ m1 = topology_m4.ms["master1"]
+ m2 = topology_m4.ms["master2"]
+ m3 = topology_m4.ms["master3"]
+ m4 = topology_m4.ms["master4"]
+ m3.stop()
+ m4.stop()
+
+ # Add over 64 tasks between master1 & 2 to try to exceed the 64 task limit
+ for i in range(1, 64):
+ cruv_task = CleanAllRUVTask(m1)
+ cruv_task.create(properties={
+ 'replica-id': str(i),
+ 'replica-base-dn': DEFAULT_SUFFIX,
+ 'replica-force-cleaning': 'no', # This forces these tasks to stick around
+ })
+ cruv_task = CleanAllRUVTask(m2)
+ cruv_task.create(properties={
+ 'replica-id': "10" + str(i),
+ 'replica-base-dn': DEFAULT_SUFFIX,
+ 'replica-force-cleaning': 'yes', # This allows the tasks to propagate
+ })
+
+ # Check the errors log for our error message in master 1
+ assert m1.searchErrorsLog('Exceeded maximum number of active CLEANALLRUV tasks')
+>>>>>>> ab24aa4cb... Issue 50538 - cleanAllRUV task limit is not enforced for replicated tasks
+
+
if __name__ == '__main__':
# Run isolated
# -s for DEBUG mode
diff --git a/ldap/servers/plugins/replication/repl5.h b/ldap/servers/plugins/replication/repl5.h
index 3c7f06f36..b06c6fbf4 100644
--- a/ldap/servers/plugins/replication/repl5.h
+++ b/ldap/servers/plugins/replication/repl5.h
@@ -80,6 +80,8 @@
#define CLEANRUV_FINISHED "finished"
#define CLEANRUV_CLEANING "cleaning"
#define CLEANRUV_NO_MAXCSN "no maxcsn"
+#define CLEANALLRUV_ID "CleanAllRUV Task"
+#define ABORT_CLEANALLRUV_ID "Abort CleanAllRUV Task"
/* DS 5.0 replication protocol error codes */
#define NSDS50_REPL_REPLICA_READY 0x00 /* Replica ready, go ahead */
@@ -784,6 +786,7 @@ void multimaster_mtnode_construct_replicas(void);
void multimaster_be_state_change(void *handle, char *be_name, int old_be_state, int new_be_state);
#define CLEANRIDSIZ 64 /* maximum number for concurrent CLEANALLRUV tasks */
+#define CLEANRID_BUFSIZ 128
typedef struct _cleanruv_data
{
@@ -815,6 +818,8 @@ int get_replica_type(Replica *r);
int replica_execute_cleanruv_task_ext(Object *r, ReplicaId rid);
void add_cleaned_rid(cleanruv_data *data, char *maxcsn);
int is_cleaned_rid(ReplicaId rid);
+int32_t check_and_set_cleanruv_task_count(ReplicaId rid);
+int32_t check_and_set_abort_cleanruv_task_count(void);
int replica_cleanall_ruv_abort(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *eAfter, int *returncode, char *returntext, void *arg);
void replica_cleanallruv_thread_ext(void *arg);
void stop_ruv_cleaning(void);
@@ -833,8 +838,6 @@ void set_cleaned_rid(ReplicaId rid);
void cleanruv_log(Slapi_Task *task, int rid, char *task_type, int sev_level, char *fmt, ...);
char *replica_cleanallruv_get_local_maxcsn(ReplicaId rid, char *base_dn);
-
-
/* replutil.c */
LDAPControl *create_managedsait_control(void);
LDAPControl *create_backend_control(Slapi_DN *sdn);
diff --git a/ldap/servers/plugins/replication/repl5_replica_config.c b/ldap/servers/plugins/replication/repl5_replica_config.c
index 62bfcf6ce..80a079784 100644
--- a/ldap/servers/plugins/replication/repl5_replica_config.c
+++ b/ldap/servers/plugins/replication/repl5_replica_config.c
@@ -30,17 +30,18 @@
#define CLEANALLRUV "CLEANALLRUV"
#define CLEANALLRUVLEN 11
#define REPLICA_RDN "cn=replica"
-#define CLEANALLRUV_ID "CleanAllRUV Task"
-#define ABORT_CLEANALLRUV_ID "Abort CleanAllRUV Task"
int slapi_log_urp = SLAPI_LOG_REPL;
-static ReplicaId cleaned_rids[CLEANRIDSIZ + 1] = {0};
-static ReplicaId pre_cleaned_rids[CLEANRIDSIZ + 1] = {0};
-static ReplicaId aborted_rids[CLEANRIDSIZ + 1] = {0};
-static Slapi_RWLock *rid_lock = NULL;
-static Slapi_RWLock *abort_rid_lock = NULL;
+static ReplicaId cleaned_rids[CLEANRID_BUFSIZ] = {0};
+static ReplicaId pre_cleaned_rids[CLEANRID_BUFSIZ] = {0};
+static ReplicaId aborted_rids[CLEANRID_BUFSIZ] = {0};
+static PRLock *rid_lock = NULL;
+static PRLock *abort_rid_lock = NULL;
static PRLock *notify_lock = NULL;
static PRCondVar *notify_cvar = NULL;
+static PRLock *task_count_lock = NULL;
+static int32_t clean_task_count = 0;
+static int32_t abort_task_count = 0;
/* Forward Declartions */
static int replica_config_add(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *entryAfter, int *returncode, char *returntext, void *arg);
@@ -67,8 +68,6 @@ static int replica_cleanallruv_send_abort_extop(Repl_Agmt *ra, Slapi_Task *task,
static int replica_cleanallruv_check_maxcsn(Repl_Agmt *agmt, char *basedn, char *rid_text, char *maxcsn, Slapi_Task *task);
static int replica_cleanallruv_replica_alive(Repl_Agmt *agmt);
static int replica_cleanallruv_check_ruv(char *repl_root, Repl_Agmt *ra, char *rid_text, Slapi_Task *task, char *force);
-static int get_cleanruv_task_count(void);
-static int get_abort_cleanruv_task_count(void);
static int replica_cleanup_task(Object *r, const char *task_name, char *returntext, int apply_mods);
static int replica_task_done(Replica *replica);
static void delete_cleaned_rid_config(cleanruv_data *data);
@@ -114,20 +113,27 @@ replica_config_init()
PR_GetError());
return -1;
}
- rid_lock = slapi_new_rwlock();
+ rid_lock = PR_NewLock();
if (rid_lock == NULL) {
slapi_log_err(SLAPI_LOG_ERR, repl_plugin_name, "replica_config_init - "
"Failed to create rid_lock; NSPR error - %d\n",
PR_GetError());
return -1;
}
- abort_rid_lock = slapi_new_rwlock();
+ abort_rid_lock = PR_NewLock();
if (abort_rid_lock == NULL) {
slapi_log_err(SLAPI_LOG_ERR, repl_plugin_name, "replica_config_init - "
"Failed to create abort_rid_lock; NSPR error - %d\n",
PR_GetError());
return -1;
}
+ task_count_lock = PR_NewLock();
+ if (task_count_lock == NULL) {
+ slapi_log_err(SLAPI_LOG_ERR, repl_plugin_name, "replica_config_init - "
+ "Failed to create task_count_lock; NSPR error - %d\n",
+ PR_GetError());
+ return -1;
+ }
if ((notify_lock = PR_NewLock()) == NULL) {
slapi_log_err(SLAPI_LOG_ERR, repl_plugin_name, "replica_config_init - "
"Failed to create notify lock; NSPR error - %d\n",
@@ -1483,12 +1489,6 @@ replica_execute_cleanall_ruv_task(Object *r, ReplicaId rid, Slapi_Task *task, co
cleanruv_log(pre_task, rid, CLEANALLRUV_ID, SLAPI_LOG_INFO, "Initiating CleanAllRUV Task...");
- if (get_cleanruv_task_count() >= CLEANRIDSIZ) {
- /* we are already running the maximum number of tasks */
- cleanruv_log(pre_task, rid, CLEANALLRUV_ID, SLAPI_LOG_ERR,
- "Exceeded maximum number of active CLEANALLRUV tasks(%d)", CLEANRIDSIZ);
- return LDAP_UNWILLING_TO_PERFORM;
- }
/*
* Grab the replica
*/
@@ -1540,6 +1540,13 @@ replica_execute_cleanall_ruv_task(Object *r, ReplicaId rid, Slapi_Task *task, co
goto fail;
}
+ if (check_and_set_cleanruv_task_count(rid) != LDAP_SUCCESS) {
+ cleanruv_log(NULL, rid, CLEANALLRUV_ID, SLAPI_LOG_ERR,
+ "Exceeded maximum number of active CLEANALLRUV tasks(%d)", CLEANRIDSIZ);
+ rc = LDAP_UNWILLING_TO_PERFORM;
+ goto fail;
+ }
+
/*
* Launch the cleanallruv thread. Once all the replicas are cleaned it will release the rid
*/
@@ -1547,6 +1554,9 @@ replica_execute_cleanall_ruv_task(Object *r, ReplicaId rid, Slapi_Task *task, co
if (data == NULL) {
cleanruv_log(pre_task, rid, CLEANALLRUV_ID, SLAPI_LOG_ERR, "Failed to allocate cleanruv_data. Aborting task.");
rc = -1;
+ PR_Lock(task_count_lock);
+ clean_task_count--;
+ PR_Unlock(task_count_lock);
goto fail;
}
data->repl_obj = r;
@@ -1629,13 +1639,13 @@ replica_cleanallruv_thread(void *arg)
int aborted = 0;
int rc = 0;
- if (!data || slapi_is_shutting_down()) {
- return; /* no data */
- }
-
/* Increase active thread count to prevent a race condition at server shutdown */
g_incr_active_threadcnt();
+ if (!data || slapi_is_shutting_down()) {
+ goto done;
+ }
+
if (data->task) {
slapi_task_inc_refcount(data->task);
slapi_log_err(SLAPI_LOG_PLUGIN, repl_plugin_name,
@@ -1682,16 +1692,13 @@ replica_cleanallruv_thread(void *arg)
slapi_task_begin(data->task, 1);
}
/*
- * Presetting the rid prevents duplicate thread creation, but allows the db and changelog to still
- * process updates from the rid.
- * set_cleaned_rid() blocks updates, so we don't want to do that... yet unless we are in force mode.
- * If we are forcing a clean independent of state of other servers for this RID we can set_cleaned_rid()
+ * We have already preset this rid, but if we are forcing a clean independent of state
+ * of other servers for this RID we can set_cleaned_rid()
*/
if (data->force) {
set_cleaned_rid(data->rid);
- } else {
- preset_cleaned_rid(data->rid);
}
+
rid_text = slapi_ch_smprintf("%d", data->rid);
csn_as_string(data->maxcsn, PR_FALSE, csnstr);
/*
@@ -1861,6 +1868,9 @@ done:
/*
* If the replicas are cleaned, release the rid
*/
+ if (slapi_is_shutting_down()) {
+ stop_ruv_cleaning();
+ }
if (!aborted && !slapi_is_shutting_down()) {
/*
* Success - the rid has been cleaned!
@@ -1879,10 +1889,9 @@ done:
} else {
cleanruv_log(data->task, data->rid, CLEANALLRUV_ID, SLAPI_LOG_INFO, "Propagated task does not delete Keep alive entry (%d).", data->rid);
}
-
clean_agmts(data);
remove_cleaned_rid(data->rid);
- cleanruv_log(data->task, data->rid, CLEANALLRUV_ID, SLAPI_LOG_INFO, "Successfully cleaned rid(%d).", data->rid);
+ cleanruv_log(data->task, data->rid, CLEANALLRUV_ID, SLAPI_LOG_INFO, "Successfully cleaned rid(%d)", data->rid);
} else {
/*
* Shutdown or abort
@@ -1915,6 +1924,10 @@ done:
slapi_ch_free_string(&data->force);
slapi_ch_free_string(&rid_text);
slapi_ch_free((void **)&data);
+ /* decrement task count */
+ PR_Lock(task_count_lock);
+ clean_task_count--;
+ PR_Unlock(task_count_lock);
g_decr_active_threadcnt();
}
@@ -2414,16 +2427,14 @@ replica_send_cleanruv_task(Repl_Agmt *agmt, cleanruv_data *clean_data)
int
is_cleaned_rid(ReplicaId rid)
{
- int i;
-
- slapi_rwlock_rdlock(rid_lock);
- for (i = 0; i < CLEANRIDSIZ && cleaned_rids[i] != 0; i++) {
+ PR_Lock(rid_lock);
+ for (size_t i = 0; i < CLEANRID_BUFSIZ; i++) {
if (rid == cleaned_rids[i]) {
- slapi_rwlock_unlock(rid_lock);
+ PR_Unlock(rid_lock);
return 1;
}
}
- slapi_rwlock_unlock(rid_lock);
+ PR_Unlock(rid_lock);
return 0;
}
@@ -2431,16 +2442,14 @@ is_cleaned_rid(ReplicaId rid)
int
is_pre_cleaned_rid(ReplicaId rid)
{
- int i;
-
- slapi_rwlock_rdlock(rid_lock);
- for (i = 0; i < CLEANRIDSIZ && pre_cleaned_rids[i] != 0; i++) {
+ PR_Lock(rid_lock);
+ for (size_t i = 0; i < CLEANRID_BUFSIZ; i++) {
if (rid == pre_cleaned_rids[i]) {
- slapi_rwlock_unlock(rid_lock);
+ PR_Unlock(rid_lock);
return 1;
}
}
- slapi_rwlock_unlock(rid_lock);
+ PR_Unlock(rid_lock);
return 0;
}
@@ -2453,14 +2462,14 @@ is_task_aborted(ReplicaId rid)
if (rid == 0) {
return 0;
}
- slapi_rwlock_rdlock(abort_rid_lock);
- for (i = 0; i < CLEANRIDSIZ && aborted_rids[i] != 0; i++) {
+ PR_Lock(abort_rid_lock);
+ for (i = 0; i < CLEANRID_BUFSIZ && aborted_rids[i] != 0; i++) {
if (rid == aborted_rids[i]) {
- slapi_rwlock_unlock(abort_rid_lock);
+ PR_Unlock(abort_rid_lock);
return 1;
}
}
- slapi_rwlock_unlock(abort_rid_lock);
+ PR_Unlock(abort_rid_lock);
return 0;
}
@@ -2469,15 +2478,14 @@ preset_cleaned_rid(ReplicaId rid)
{
int i;
- slapi_rwlock_wrlock(rid_lock);
- for (i = 0; i < CLEANRIDSIZ; i++) {
+ PR_Lock(rid_lock);
+ for (i = 0; i < CLEANRID_BUFSIZ && pre_cleaned_rids[i] != rid; i++) {
if (pre_cleaned_rids[i] == 0) {
pre_cleaned_rids[i] = rid;
- pre_cleaned_rids[i + 1] = 0;
break;
}
}
- slapi_rwlock_unlock(rid_lock);
+ PR_Unlock(rid_lock);
}
/*
@@ -2490,14 +2498,13 @@ set_cleaned_rid(ReplicaId rid)
{
int i;
- slapi_rwlock_wrlock(rid_lock);
- for (i = 0; i < CLEANRIDSIZ; i++) {
+ PR_Lock(rid_lock);
+ for (i = 0; i < CLEANRID_BUFSIZ && cleaned_rids[i] != rid; i++) {
if (cleaned_rids[i] == 0) {
cleaned_rids[i] = rid;
- cleaned_rids[i + 1] = 0;
}
}
- slapi_rwlock_unlock(rid_lock);
+ PR_Unlock(rid_lock);
}
/*
@@ -2569,15 +2576,14 @@ add_aborted_rid(ReplicaId rid, Replica *r, char *repl_root)
int rc;
int i;
- slapi_rwlock_wrlock(abort_rid_lock);
- for (i = 0; i < CLEANRIDSIZ; i++) {
+ PR_Lock(abort_rid_lock);
+ for (i = 0; i < CLEANRID_BUFSIZ; i++) {
if (aborted_rids[i] == 0) {
aborted_rids[i] = rid;
- aborted_rids[i + 1] = 0;
break;
}
}
- slapi_rwlock_unlock(abort_rid_lock);
+ PR_Unlock(abort_rid_lock);
/*
* Write the rid to the config entry
*/
@@ -2620,21 +2626,24 @@ delete_aborted_rid(Replica *r, ReplicaId rid, char *repl_root, int skip)
char *data;
char *dn;
int rc;
- int i;
if (r == NULL)
return;
if (skip) {
/* skip the deleting of the config, and just remove the in memory rid */
- slapi_rwlock_wrlock(abort_rid_lock);
- for (i = 0; i < CLEANRIDSIZ && aborted_rids[i] != rid; i++)
- ; /* found rid, stop */
- for (; i < CLEANRIDSIZ; i++) {
- /* rewrite entire array */
- aborted_rids[i] = aborted_rids[i + 1];
- }
- slapi_rwlock_unlock(abort_rid_lock);
+ ReplicaId new_abort_rids[CLEANRID_BUFSIZ] = {0};
+ int32_t idx = 0;
+
+ PR_Lock(abort_rid_lock);
+ for (size_t i = 0; i < CLEANRID_BUFSIZ; i++) {
+ if (aborted_rids[i] != rid) {
+ new_abort_rids[idx] = aborted_rids[i];
+ idx++;
+ }
+ }
+ memcpy(aborted_rids, new_abort_rids, sizeof(new_abort_rids));
+ PR_Unlock(abort_rid_lock);
} else {
/* only remove the config, leave the in-memory rid */
dn = replica_get_dn(r);
@@ -2792,27 +2801,31 @@ bail:
void
remove_cleaned_rid(ReplicaId rid)
{
- int i;
- /*
- * Remove this rid, and optimize the array
- */
- slapi_rwlock_wrlock(rid_lock);
+ ReplicaId new_cleaned_rids[CLEANRID_BUFSIZ] = {0};
+ ReplicaId new_pre_cleaned_rids[CLEANRID_BUFSIZ] = {0};
+ size_t idx = 0;
+
+ PR_Lock(rid_lock);
- for (i = 0; i < CLEANRIDSIZ && cleaned_rids[i] != rid; i++)
- ; /* found rid, stop */
- for (; i < CLEANRIDSIZ; i++) {
- /* rewrite entire array */
- cleaned_rids[i] = cleaned_rids[i + 1];
+ for (size_t i = 0; i < CLEANRID_BUFSIZ; i++) {
+ if (cleaned_rids[i] != rid) {
+ new_cleaned_rids[idx] = cleaned_rids[i];
+ idx++;
+ }
}
+ memcpy(cleaned_rids, new_cleaned_rids, sizeof(new_cleaned_rids));
+
/* now do the preset cleaned rids */
- for (i = 0; i < CLEANRIDSIZ && pre_cleaned_rids[i] != rid; i++)
- ; /* found rid, stop */
- for (; i < CLEANRIDSIZ; i++) {
- /* rewrite entire array */
- pre_cleaned_rids[i] = pre_cleaned_rids[i + 1];
+ idx = 0;
+ for (size_t i = 0; i < CLEANRID_BUFSIZ; i++) {
+ if (pre_cleaned_rids[i] != rid) {
+ new_pre_cleaned_rids[idx] = pre_cleaned_rids[i];
+ idx++;
+ }
}
+ memcpy(pre_cleaned_rids, new_pre_cleaned_rids, sizeof(new_pre_cleaned_rids));
- slapi_rwlock_unlock(rid_lock);
+ PR_Unlock(rid_lock);
}
/*
@@ -2840,16 +2853,6 @@ replica_cleanall_ruv_abort(Slapi_PBlock *pb __attribute__((unused)),
char *ridstr = NULL;
int rc = SLAPI_DSE_CALLBACK_OK;
- if (get_abort_cleanruv_task_count() >= CLEANRIDSIZ) {
- /* we are already running the maximum number of tasks */
- PR_snprintf(returntext, SLAPI_DSE_RETURNTEXT_SIZE,
- "Exceeded maximum number of active ABORT CLEANALLRUV tasks(%d)",
- CLEANRIDSIZ);
- cleanruv_log(task, -1, ABORT_CLEANALLRUV_ID, SLAPI_LOG_ERR, "%s", returntext);
- *returncode = LDAP_OPERATIONS_ERROR;
- return SLAPI_DSE_CALLBACK_ERROR;
- }
-
/* allocate new task now */
task = slapi_new_task(slapi_entry_get_ndn(e));
@@ -2934,6 +2937,16 @@ replica_cleanall_ruv_abort(Slapi_PBlock *pb __attribute__((unused)),
*/
certify_all = "no";
}
+
+ if (check_and_set_abort_cleanruv_task_count() != LDAP_SUCCESS) {
+ /* we are already running the maximum number of tasks */
+ PR_snprintf(returntext, SLAPI_DSE_RETURNTEXT_SIZE,
+ "Exceeded maximum number of active ABORT CLEANALLRUV tasks(%d)",
+ CLEANRIDSIZ);
+ cleanruv_log(task, -1, ABORT_CLEANALLRUV_ID, SLAPI_LOG_ERR, "%s", returntext);
+ *returncode = LDAP_UNWILLING_TO_PERFORM;
+ goto out;
+ }
/*
* Create payload
*/
@@ -3142,6 +3155,9 @@ done:
slapi_ch_free_string(&data->certify);
slapi_sdn_free(&data->sdn);
slapi_ch_free((void **)&data);
+ PR_Lock(task_count_lock);
+ abort_task_count--;
+ PR_Unlock(task_count_lock);
g_decr_active_threadcnt();
}
@@ -3493,36 +3509,43 @@ replica_cleanallruv_check_ruv(char *repl_root, Repl_Agmt *agmt, char *rid_text,
return rc;
}
-static int
-get_cleanruv_task_count(void)
+/*
+ * Before starting a cleanAllRUV task make sure there are not
+ * too many task threads already running. If everything is okay
+ * also pre-set the RID now so rebounding extended ops do not
+ * try to clean it over and over.
+ */
+int32_t
+check_and_set_cleanruv_task_count(ReplicaId rid)
{
- int i, count = 0;
+ int32_t rc = 0;
- slapi_rwlock_wrlock(rid_lock);
- for (i = 0; i < CLEANRIDSIZ; i++) {
- if (pre_cleaned_rids[i] != 0) {
- count++;
- }
+ PR_Lock(task_count_lock);
+ if (clean_task_count >= CLEANRIDSIZ) {
+ rc = -1;
+ } else {
+ clean_task_count++;
+ preset_cleaned_rid(rid);
}
- slapi_rwlock_unlock(rid_lock);
+ PR_Unlock(task_count_lock);
- return count;
+ return rc;
}
-static int
-get_abort_cleanruv_task_count(void)
+int32_t
+check_and_set_abort_cleanruv_task_count(void)
{
- int i, count = 0;
+ int32_t rc = 0;
- slapi_rwlock_wrlock(rid_lock);
- for (i = 0; i < CLEANRIDSIZ; i++) {
- if (aborted_rids[i] != 0) {
- count++;
+ PR_Lock(task_count_lock);
+ if (abort_task_count > CLEANRIDSIZ) {
+ rc = -1;
+ } else {
+ abort_task_count++;
}
- }
- slapi_rwlock_unlock(rid_lock);
+ PR_Unlock(task_count_lock);
- return count;
+ return rc;
}
/*
diff --git a/ldap/servers/plugins/replication/repl_extop.c b/ldap/servers/plugins/replication/repl_extop.c
index 68e2544b4..0c2abb6d5 100644
--- a/ldap/servers/plugins/replication/repl_extop.c
+++ b/ldap/servers/plugins/replication/repl_extop.c
@@ -1393,6 +1393,12 @@ multimaster_extop_abort_cleanruv(Slapi_PBlock *pb)
rc = LDAP_OPERATIONS_ERROR;
goto out;
}
+ if (check_and_set_abort_cleanruv_task_count() != LDAP_SUCCESS) {
+ cleanruv_log(NULL, rid, CLEANALLRUV_ID, SLAPI_LOG_ERR,
+ "Exceeded maximum number of active abort CLEANALLRUV tasks(%d)", CLEANRIDSIZ);
+ rc = LDAP_UNWILLING_TO_PERFORM;
+ goto out;
+ }
/*
* Prepare the abort data
*/
@@ -1499,6 +1505,7 @@ multimaster_extop_cleanruv(Slapi_PBlock *pb)
if (force == NULL) {
force = "no";
}
+
maxcsn = csn_new();
csn_init_by_string(maxcsn, csnstr);
/*
@@ -1535,13 +1542,21 @@ multimaster_extop_cleanruv(Slapi_PBlock *pb)
goto free_and_return;
}
+ if (check_and_set_cleanruv_task_count((ReplicaId)rid) != LDAP_SUCCESS) {
+ cleanruv_log(NULL, rid, CLEANALLRUV_ID, SLAPI_LOG_ERR,
+ "Exceeded maximum number of active CLEANALLRUV tasks(%d)", CLEANRIDSIZ);
+ rc = LDAP_UNWILLING_TO_PERFORM;
+ goto free_and_return;
+ }
+
if (replica_get_type(r) != REPLICA_TYPE_READONLY) {
/*
* Launch the cleanruv monitoring thread. Once all the replicas are cleaned it will release the rid
*
* This will also release mtnode_ext->replica
*/
- slapi_log_err(SLAPI_LOG_INFO, repl_plugin_name, "multimaster_extop_cleanruv - CleanAllRUV Task - Launching cleanAllRUV thread...\n");
+
+ cleanruv_log(NULL, rid, CLEANALLRUV_ID, SLAPI_LOG_ERR, "Launching cleanAllRUV thread...\n");
data = (cleanruv_data *)slapi_ch_calloc(1, sizeof(cleanruv_data));
if (data == NULL) {
slapi_log_err(SLAPI_LOG_ERR, repl_plugin_name, "multimaster_extop_cleanruv - CleanAllRUV Task - Failed to allocate "
@@ -1635,7 +1650,7 @@ free_and_return:
ber_printf(resp_bere, "{s}", CLEANRUV_ACCEPTED);
ber_flatten(resp_bere, &resp_bval);
slapi_pblock_set(pb, SLAPI_EXT_OP_RET_VALUE, resp_bval);
- slapi_send_ldap_result(pb, LDAP_SUCCESS, NULL, NULL, 0, NULL);
+ slapi_send_ldap_result(pb, rc, NULL, NULL, 0, NULL);
/* resp_bere */
if (NULL != resp_bere) {
ber_free(resp_bere, 1);
--
2.21.0

39
SOURCES/0005-Issue-49624-modrdn-silently-fails-if-DB-deadlock-occ.patch
View file

@ -1,39 +0,0 @@
From f50dfbb61224e6a9516b93cd3d3957c1fde4798e Mon Sep 17 00:00:00 2001
From: Mark Reynolds <mreynolds@redhat.com>
Date: Thu, 22 Aug 2019 10:26:24 -0400
Subject: [PATCH] Issue 49624 - modrdn silently fails if DB deadlock occurs
Bug Description:
If a DB Deadlock error occurs during a modrdn operation the entry
cache gets updated (corrupted), but the update is not applied to
the database.
Fix Description:
Looks like there was a copy & paste error, and the wrong attribute
was updated during the retry of the modrdn operation.
relates: https://pagure.io/389-ds-base/issue/49624
Reviewed by: lkrispenz (Thanks!)
---
ldap/servers/slapd/back-ldbm/ldbm_modrdn.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/ldap/servers/slapd/back-ldbm/ldbm_modrdn.c b/ldap/servers/slapd/back-ldbm/ldbm_modrdn.c
index 65610d613..433ed88fb 100644
--- a/ldap/servers/slapd/back-ldbm/ldbm_modrdn.c
+++ b/ldap/servers/slapd/back-ldbm/ldbm_modrdn.c
@@ -251,7 +251,7 @@ ldbm_back_modrdn(Slapi_PBlock *pb)
slapi_pblock_get(pb, SLAPI_MODRDN_NEWSUPERIOR_SDN, &dn_newsuperiordn);
slapi_sdn_free(&dn_newsuperiordn);
slapi_pblock_set(pb, SLAPI_MODRDN_NEWSUPERIOR_SDN, orig_dn_newsuperiordn);
- orig_dn_newsuperiordn = slapi_sdn_dup(orig_dn_newsuperiordn);
+ dn_newsuperiordn = slapi_sdn_dup(orig_dn_newsuperiordn);
/* must duplicate ec before returning it to cache,
* which could free the entry. */
if ((tmpentry = backentry_dup(original_entry ? original_entry : ec)) == NULL) {
--
2.21.0

153
SOURCES/0006-Issue-50572-After-running-cl-dump-dbdir-cldb-ldif.do.patch
View file

@ -1,153 +0,0 @@
From d8935139d377ad75be4242db7d3194f3706dc44a Mon Sep 17 00:00:00 2001
From: Simon Pichugin <spichugi@redhat.com>
Date: Thu, 29 Aug 2019 15:51:56 +0200
Subject: [PATCH] Issue 50572 - After running cl-dump dbdir/cldb/*ldif.done are
not deleted
Description: By default, remove ldif.done files after running cl-dump.
Add an option '-l' which allows keep the files.
Update man files.
https://pagure.io/389-ds-base/issue/50572
Reviewed by: firstyear, mreynolds (Thanks!)
---
ldap/admin/src/scripts/cl-dump.pl | 23 +++++++++++++++--------
man/man1/cl-dump.1 | 11 +++++++----
2 files changed, 22 insertions(+), 12 deletions(-)
diff --git a/ldap/admin/src/scripts/cl-dump.pl b/ldap/admin/src/scripts/cl-dump.pl
index f4ad5dd33..2e7f20413 100755
--- a/ldap/admin/src/scripts/cl-dump.pl
+++ b/ldap/admin/src/scripts/cl-dump.pl
@@ -5,7 +5,7 @@
# All rights reserved.
#
# License: GPL (version 3 or any later version).
-# See LICENSE for details.
+# See LICENSE for details.
# END COPYRIGHT BLOCK
###################################################################################
#
@@ -13,7 +13,7 @@
#
# SYNOPSIS:
# cl-dump.pl [-h host] [-p port] [-D bind-dn] -w bind-password | -P bind-cert
-# [-r replica-roots] [-o output-file] [-c] [-v]
+# [-r replica-roots] [-o output-file] [-c] [-l] [-v]
#
# cl-dump.pl -i changelog-ldif-file-with-base64encoding [-o output-file] [-c]
#
@@ -22,7 +22,7 @@
#
# OPTIONS:
# -c Dump and interpret CSN only. This option can be used with or
-# without -i option.
+# without -i option.
#
# -D bind-dn
# Directory server's bind DN. Default to "cn=Directory Manager" if
@@ -32,6 +32,8 @@
# Directory server's host. Default to the server where the script
# is running.
#
+# -l Preserve generated ldif.done files from changelogdir
+#
# -i changelog-ldif-file-with-base64encoding
# If you already have a ldif-like changelog, but the changes
# in that file are encoded, you may use this option to
@@ -68,7 +70,7 @@
# all of this nonsense can be omitted if the mozldapsdk and perldap are
# installed in the operating system locations (e.g. /usr/lib /usr/lib/perl5)
-$usage="Usage: $0 [-h host] [-p port] [-D bind-dn] [-w bind-password | -P bind-cert] [-r replica-roots] [-o output-file] [-c] [-v]\n\n $0 -i changelog-ldif-file-with-base64encoding [-o output-file] [-c]\n";
+$usage="Usage: $0 [-h host] [-p port] [-D bind-dn] [-w bind-password | -P bind-cert] [-r replica-roots] [-o output-file] [-c] [-l] [-v]\n\n $0 -i changelog-ldif-file-with-base64encoding [-o output-file] [-c]\n";
use Getopt::Std; # Parse command line arguments
use Mozilla::LDAP::Conn; # LDAP module for Perl
@@ -86,7 +88,7 @@ $version = "Directory Server Changelog Dump - Version 1.0";
$| = 1;
# Check for legal options
- if (!getopts('h:p:D:w:P:r:o:cvi:')) {
+ if (!getopts('h:p:D:w:P:r:o:clvi:')) {
print $usage;
exit -1;
}
@@ -123,7 +125,7 @@ sub validateArgs
if ($opt_o && ! open (OUTPUT, ">$opt_o")) {
print "Can't create output file $opt_o\n";
$rc = -1;
- }
+ }
# Open STDOUT if option -o is missing
open (OUTPUT, ">-") if !$opt_o;
@@ -194,10 +196,15 @@ sub cl_dump_and_decode
else {
&cl_decode ($_);
}
- # Test op -M doesn't work well so we use rename
+ # Test op -M doesn't work well so we use rename/remove
# here to avoid reading the same ldif file more
# than once.
- rename ($ldif, "$ldif.done");
+ if ($opt_l) {
+ rename ($ldif, "$ldif.done");
+ } else {
+ # Remove the file - default behaviou when '-l' is not specified
+ unlink ($ldif)
+ }
}
&print_header ($replica, "Not Found") if !$gotldif;
}
diff --git a/man/man1/cl-dump.1 b/man/man1/cl-dump.1
index db736aca9..fbb836a72 100644
--- a/man/man1/cl-dump.1
+++ b/man/man1/cl-dump.1
@@ -20,7 +20,7 @@ cl-dump \- Dump and decode Directory Server replication change log
.SH SYNOPSIS
.B cl\-dump
[\fI\-h host\fR] [\fI\-p port\fR] [\fI\-D bind\(hydn\fR] \-w bind\(hypassword | \-P bind\(hycert
- [\fI\-r replica\(hyroots\fR] [\fI\-o output\(hyfile\fR] [\fI\-c\fR] [\fI\-v\fR]
+ [\fI\-r replica\(hyroots\fR] [\fI\-o output\(hyfile\fR] [\fI\-c\fR] [\fI\-l\fR] [\fI\-v\fR]
.PP
.B cl\-dump
@@ -30,12 +30,12 @@ cl-dump \- Dump and decode Directory Server replication change log
Dump and decode Directory Server replication change log
.PP
.\" TeX users may be more comfortable with the \fB<whatever>\fP and
-.\" \fI<whatever>\fP escape sequences to invode bold face and italics,
+.\" \fI<whatever>\fP escape sequences to invode bold face and italics,
.\" respectively.
.SH OPTIONS
A summary of options is included below.
.TP
-.B \-c
+.B \-c
Dump and interpret CSN only. This option can be used with or
without \-i option.
.TP
@@ -47,6 +47,9 @@ the option is omitted.
Directory server's host. Default to the server where the script
is running.
.TP
+.B \-l
+Preserve generated ldif.done files from changelogdir
+.TP
.B \-i changelog\(hyldif\(hyfile\(hywith\(hybase64encoding
If you already have a ldif-like changelog, but the changes
in that file are encoded, you may use this option to
@@ -66,7 +69,7 @@ Specify replica roots whose changelog you want to dump. The replica
roots may be separated by comma. All the replica roots would be
dumped if the option is omitted.
.TP
-.B \-v
+.B \-v
Print the version of this script.
.TP
.B \-w bind\(hypassword
--
2.21.0

27
SOURCES/0007-Issue-50538-Fix-cherry-pick-error.patch
View file

@ -1,27 +0,0 @@
From 5fee4d79db94684d8093501fde3422ad34ac2716 Mon Sep 17 00:00:00 2001
From: Mark Reynolds <mreynolds@redhat.com>
Date: Tue, 3 Sep 2019 13:40:27 -0400
Subject: [PATCH] Issue 50538 - Fix cherry-pick error
Description: Remove cherry-=pick error formating
relates: https://pagure.io/389-ds-base/issue/50538
---
dirsrvtests/tests/suites/replication/cleanallruv_test.py | 1 -
1 file changed, 1 deletion(-)
diff --git a/dirsrvtests/tests/suites/replication/cleanallruv_test.py b/dirsrvtests/tests/suites/replication/cleanallruv_test.py
index 43801dd52..caf214b19 100644
--- a/dirsrvtests/tests/suites/replication/cleanallruv_test.py
+++ b/dirsrvtests/tests/suites/replication/cleanallruv_test.py
@@ -899,7 +899,6 @@ def test_max_tasks(topology_m4):
# Check the errors log for our error message in master 1
assert m1.searchErrorsLog('Exceeded maximum number of active CLEANALLRUV tasks')
->>>>>>> ab24aa4cb... Issue 50538 - cleanAllRUV task limit is not enforced for replicated tasks
if __name__ == '__main__':
--
2.21.0

56
SPECS/389-ds-base.spec
View file

@ -38,8 +38,8 @@
Summary: 389 Directory Server (%{variant})
Name: 389-ds-base
Version: 1.3.10.1
Release: %{?relprefix}2%{?prerel}%{?dist}
Version: 1.3.10.2
Release: %{?relprefix}1%{?prerel}%{?dist}
License: GPLv3+
URL: https://www.port389.org/
Group: System Environment/Daemons
@ -145,14 +145,7 @@ Requires: gperftools-libs
Source0: https://releases.pagure.org/389-ds-base/%{name}-%{version}%{?prerel}.tar.bz2
Source1: %{name}-git.sh
Source2: %{name}-devel.README
Patch00: 0000-CVE-2019-14824-BZ-1748199-deref-plugin-displays-rest.patch
Patch01: 0001-Issue-50525-nsslapd-defaultnamingcontext-does-not-ch.patch
Patch02: 0002-Issue-50530-Directory-Server-not-RFC-4511-compliant-.patch
Patch03: 0003-Issue-50529-LDAP-server-returning-PWP-controls-in-di.patch
Patch04: 0004-Issue-50538-cleanAllRUV-task-limit-is-not-enforced-f.patch
Patch05: 0005-Issue-49624-modrdn-silently-fails-if-DB-deadlock-occ.patch
Patch06: 0006-Issue-50572-After-running-cl-dump-dbdir-cldb-ldif.do.patch
Patch07: 0007-Issue-50538-Fix-cherry-pick-error.patch
%description
389 Directory Server is an LDAPv3 compliant server. The base package includes
@ -505,6 +498,49 @@ fi
%{_sysconfdir}/%{pkgname}/dirsrvtests
%changelog
* Mon Mar 16 2020 Mark Reynolds <mreynolds@redhat.com> - 1.3.10.2-1
- Bump version to 1.3.10.2-1
- Resolves: Bug 1515319 - nsDS5ReplicaId cant be set to the old value it had before
- Resolves: Bug 1700987 - 389-base-ds expected file permissions in package don't match final runtime permissions
- Resolves: Bug 1762901 - cenotaph errors on modrdn operations
- Resolves: Bug 1772616 - Typo in the replication debug message "error 0 for oparation 561"
- Resolves: Bug 1781276 - Regression: NSS has interop problems as server when using limited cipher list
- Resolves: Bug 1787921 - Crash on startup: Bus error in __env_faultmem.isra.1.part.2
- Resolves: Bug 1759142 - No error returned when adding an entry matching filters for a non existing automember group
- Resolves: Bug 1763365 - ns-slapd is crashing while restarting ipactl
- Resolves: Bug 1769418 - Several memory leaks reported by Valgrind for 389-ds 1.3.9.1-10
- Resolves: Bug 1775165 - ldclt core dumped when run with -e genldif option
- Resolves: Bug 1796558 - Memory leak in ACI using IP subject
- Resolves: Bug 1769296 - cl-dump exit code is 0 even if command fails with invalid arguments
* Mon Mar 2 2020 Mark Reynolds <mreynolds@redhat.com> - 1.3.10.1-7
- Bump version to 1.3.10.1-7
- Resolves: Bug 1803023 - Several memory leaks reported by Valgrind (fix regression)
* Mon Mar 2 2020 Mark Reynolds <mreynolds@redhat.com> - 1.3.10.1-6
- Bump version to 1.3.10.1-6
- Resolves: Bug 1801694 - ns-slapd is crashing while restarting ipactl
- Resolves: Bug 1803023 - Several memory leaks reported by Valgrind for 389-ds 1.3.9.1-10
- Resolves: Bug 1803052 - Memory leak in ACI using IP subject
- Resolves: Bug 1801703 - Regression: NSS has interop problems as server when using limited cipher list
- Resolves: Bug 1809160 - Entry cache contention during base search
* Fri Feb 7 2020 Mark Reynolds <mreynolds@redhat.com> - 1.3.10.1-5
- Bump version to 1.3.10.1-5
- Resolves: Bug 1744623 - DB Deadlock on modrdn appears to corrupt database and entry cache(cont)
* Fri Oct 25 2019 Mark Reynolds <mreynolds@redhat.com> - 1.3.10.1-4
- Bump version to 1.3.10.1-4
- Resolves: Bug 1765106 - ipa-server-install is failing with ipapython.admintool: ERROR failed to create DS instance Command
* Thu Oct 17 2019 Mark Reynolds <mreynolds@redhat.com> - 1.3.10.1-3
- Bump version to 1.3.10.1-3
- Resolves: Bug 1756182 - ns-slapd crash on concurrent SASL BINDs, connection_call_io_layer_callbacks must hold hold c_mutex
- Resolves: Bug 1749236 - etime displayed has an order of magnitude 10 times smaller than it should be
- Resolves: Bug 1758109 - AddressSanitizer: heap-use-after-free in import_free_job
- Resolves: Bug 1676948 - After audit log file is rotated, DS version string is logged after each update
- Resolves: Bug 1749595 - Extremely slow LDIF import with ldif2db
* Tue Sep 3 2019 Mark Reynolds <mreynolds@redhat.com> - 1.3.10.1-2
- Bump version to 1.3.10.1-2
- Resolves: Bug 1748199 - EMBARGOED CVE-2019-14824 389-ds-base: 389-ds and IDM: allows authenticated unprivileged user to retrieve content of userPassword field for any user