[Bug][Manager] There is a problem with the deletion process of the data flow group.

[flowers-59f]

What happened

Description

When deleting the data flow group, the related data in the tables schedule_config and inlong_stream did not perform the logical deletion correctly.

Phenomenon

Create a data flow group

Relevant data



Delete this data flow group




It can be seen that the relevant data under the "schedule_config" and "inlong_stream" tables have not correctly performed logical deletion, because the "is_deleted" field of the relevant data was not updated.
This issue will cause problems when creating a data flow group with the same name again after deleting an existing data flow group.This is also the situation in which I discovered this problem.

What you expected to happen

I think the main problem lies in InlongGroupServiceImpl.delete(...).
The deletion process is roughly as follows:
InlongGroupController.delete(...)
->InlongGroupProcessService.deleteProcess(...)
->InlongGroupProcessService.invokeDeleteProcess(...)
->workflowService.start(ProcessName.DELETE_GROUP_PROCESS, ...);
This workflow will create and invoke UpdateGroupListener and updateGroupCompleteListener (The UpdateGroupFailedListener will only be invoked when the pre-deletion operation fails).
In the UpdateGroupListener, the status of the group was updated.

case DELETE:  
    groupService.updateStatus(groupId, GroupStatus.CONFIG_DELETING.getCode(), operator);

After the pre-deletion operation is completed, the updateGroupCompleteListener deletes the relevant information and updates the status.

case DELETE:  
    // delete process completed, then delete the group info  
    groupService.delete(groupId, operator);

There are certain issues with the groupService.delete(...) .

if (GroupStatus.allowedDeleteSubInfos(GroupStatus.forCode(entity.getStatus()))) {  
    streamService.logicDeleteAll(groupId, operator);  
}

public static boolean allowedDeleteSubInfos(GroupStatus status) {  
    return status == GroupStatus.TO_BE_SUBMIT  
            || status == GroupStatus.APPROVE_REJECTED  
            || status == GroupStatus.CONFIG_DELETED;  
}

At this point, the status of the group is CONFIG_DELETING, which is not within the permitted states. Therefore, the streamService.logicDeleteAll(...) method will not be executed.

Then the status of InlongGroupEntity was updated.

entity.setIsDeleted(entity.getId());  
entity.setStatus(GroupStatus.CONFIG_DELETED.getCode());  
entity.setModifier(operator);  
int rowCount = groupMapper.updateByIdentifierSelective(entity);  
if (rowCount != InlongConstants.AFFECTED_ONE_ROW) {  
    LOGGER.error("inlong group has already updated for groupId={} curVersion={}", groupId, entity.getVersion());  
    throw new BusinessException(ErrorCodeEnum.CONFIG_EXPIRED);  
}

However, this update operation should be carried out after the completion of the latter two deletion operations.

// logically delete the associated extension info  
groupExtMapper.logicDeleteAllByGroupId(groupId);  
  
// remove schedule  
if (DATASYNC_OFFLINE_MODE.equals(entity.getInlongGroupMode())) {  
    try {  
        scheduleOperator.deleteByGroupIdOpt(entity.getInlongGroupId(), operator);  
    } catch (Exception e) {  
        LOGGER.warn("failed to delete schedule info for groupId={}, error msg: {}", groupId, e.getMessage());  
    }  
}

Because the corresponding SQL for groupExtMapper.logicDeleteAllByGroupId(...) is as follows

<update id="logicDeleteAllByGroupId">  
    update inlong_group_ext  
    set is_deleted = id    where inlong_group_id = #{groupId, jdbcType=VARCHAR}      
    and is_deleted = 0
</update>

scheduleOperator.deleteByGroupIdOpt(...) will eventually call ScheduleServiceImpl.deleteByGroupId(...). The SQL statement used to query the relevant scheduleEntity is as follows:

<select id="selectByGroupId" parameterType="java.lang.String" resultMap="BaseResultMap">  
    select  
    <include refid="Base_Column_List"/>  
    from schedule_config  
    where inlong_group_id = #{inlongGroupId,jdbcType=VARCHAR}    
    and is_deleted = 0
</select>

Both of the above SQL statements require that is_deleted = 0, but this value has already been updated earlier.

How to reproduce

1. Create a data flow group
2. Delete it and check the database
3. You can also create another data stream group with the same name, it will result in an error.

Environment

No response

InLong version

2.3.0

InLong Component

InLong Manager

Are you willing to submit PR?

• Yes, I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct