275 Commits

Author SHA1 Message Date
265813e6f7 Merge branch 'feat/training_260202' of https://kamco.git.gs.dabeeo.com/MVPTeam/kamco-train-api into feat/training_260202 2026-03-10 14:19:40 +09:00
8190a6e9c8 unzip 2026-03-10 14:19:22 +09:00
e9f8bb37fa spotless 적용 2026-03-03 23:06:45 +09:00
f3c822587f spotless 적용 2026-03-03 23:02:53 +09:00
335f0dbb9b spotless 적용 2026-03-03 23:01:22 +09:00
69eaba1a83 하드링크 수정 2026-03-03 22:51:10 +09:00
365ad81cad 리커버리 삭제 2026-02-28 01:24:34 +09:00
9dfa54fbf9 리커버리 추가 2026-02-28 01:01:38 +09:00
12f6bb7154 하드링크 수정 2026-02-27 23:31:04 +09:00
aa3af4e9d0 하드링크 로그 추가 2026-02-27 23:12:00 +09:00
7ca37bf1e4 하드링크 로그 추가 2026-02-27 22:51:27 +09:00
901dde066d 하이파라미터 상세조회 수정 2026-02-24 16:54:58 +09:00
cb0a38274a val_interval 기본값 1로 수정 2026-02-24 16:24:16 +09:00
b8194df9ae 학습 실패여부 확인 기능 추가 2026-02-24 15:43:35 +09:00
7c5f07683e 학습 실패여부 확인 기능 추가 2026-02-24 15:10:48 +09:00
159fb281d4 최근 사용일시 업데이트 2026-02-23 15:40:24 +09:00
97192ff811 최근 사용일시 업데이트 2026-02-23 15:39:25 +09:00
4f3fb675be 하이퍼 파라미터 사용회수 카운트 기능 추가 및 조회 수정 2026-02-23 15:37:28 +09:00
e6caea05b3 하이퍼 파라미터 사용회수 카운트 기능 추가 및 조회 수정 2026-02-23 15:19:40 +09:00
fd63824edc 하이퍼 파라미터 미사용 컬럼제거, 사용횟수 컬럼 추가 2026-02-23 14:30:29 +09:00
8a44df26b8 하이퍼 파라미터 상세조회 삭제여부 조건 제거 2026-02-23 14:17:34 +09:00
cb97c5e59e 하이퍼 파라미터 dto 주석 수정 2026-02-23 14:13:25 +09:00
8f75b16dc6 학습실행 주석 추가 2026-02-23 12:30:54 +09:00
c2978e41c2 전이학습 상세 수정 2026-02-20 18:34:32 +09:00
07429dbe8e 전이학습 상세 수정 2026-02-20 18:22:19 +09:00
83859bb9fe 전이학습 상세 - before dataset 추가 2026-02-20 16:05:29 +09:00
564a99448c best epoch 파일 선택 수정 2026-02-20 15:41:34 +09:00
38ae6e5575 best epoch 파일 선택 수정 2026-02-20 15:31:33 +09:00
40fe98ae0c best epoch 파일 선택 수정 2026-02-20 15:15:12 +09:00
255ff10a56 Merge remote-tracking branch 'origin/feat/training_260202' into feat/training_260202 2026-02-20 14:30:42 +09:00
f674f73330 중복 수정 제거 2026-02-20 14:30:34 +09:00
db2bc32e7d test metrics insert 로직 수정 2026-02-20 14:24:23 +09:00
37786a1e44 선택한 테스트 에폭 로그 추가 2026-02-20 14:13:49 +09:00
901ea83fb7 test 선택한 에폭 log 확인 추가 2026-02-20 14:13:22 +09:00
832e1b5681 tmp 하드링크 수정 2026-02-20 13:36:48 +09:00
4f16355cda tmp 하드링크 수정 2026-02-20 12:29:57 +09:00
df46a8f79f Merge remote-tracking branch 'origin/feat/training_260202' into feat/training_260202 2026-02-20 12:21:50 +09:00
fcd48831c5 tmp 하드링크 수정 2026-02-20 12:21:41 +09:00
62c9d73b94 test json 수정 2026-02-20 12:20:15 +09:00
68c0e634c5 ing-cnt 로직에 step2도 추가, transactional 2026-02-20 12:05:20 +09:00
ad421e3c74 비밀번호 변경 security 로직 수정 2026-02-20 11:36:21 +09:00
46db1512a6 test 실행 시 회차별 데이터 적재하기 2026-02-19 18:18:12 +09:00
2034a8fcb2 LogErrorLevel -> CodeExpose 추가 2026-02-19 17:35:15 +09:00
bf212842d8 모델학습관리 > 모델별 진행 상황 API 추가 2026-02-19 17:17:11 +09:00
d2ca94ea55 모델학습관리 > 목록 API 메모,작성자 추가로 인한 수정 2026-02-19 15:34:18 +09:00
5ddf6dfeeb 모델학습 2단계 패키징 시작,종료일시,상태 로직 추가 2026-02-19 14:43:14 +09:00
5e13c0b396 공통코드 common-code 로 prefix 변경 2026-02-19 11:38:56 +09:00
435f60dcac 로그관리 로직 커밋 2026-02-19 11:13:40 +09:00
5f5eabca19 압축해제 시, 동일 폴더가 있으면 삭제 후 재업로드 2026-02-18 16:36:46 +09:00
413631840f 학습데이터 다운로드 security 제외하기 2026-02-18 16:29:28 +09:00
c7f63d1ad1 압축 해제한 폴더의 갯수 맞는지 log 찍기 + 갯수 맞지 않으면 exception 리턴 2026-02-18 16:22:32 +09:00
7529d23488 업로드 시 uid로 중복체크 -> 삭제인 row는 제외하기 2026-02-18 15:40:14 +09:00
cb3e51d712 업로드 시 exception 메세지 처리, 에폭 10 이상으로 실행되게 수정 2026-02-18 15:28:29 +09:00
99a4597b5f train 결과 +1 했던 거 제거하기 2026-02-18 14:50:53 +09:00
d9da0d4610 1단계 실행 시, 시작시간 update 추가 2026-02-18 13:05:59 +09:00
22c481556c 하이퍼 파라미터 수정 2026-02-13 15:00:30 +09:00
0798b352c7 하이퍼 파라미터 수정 2026-02-13 14:46:59 +09:00
5b074bdb81 하이퍼 파라미터 수정 2026-02-13 14:42:39 +09:00
28919345c2 하이퍼 파라미터 수정 2026-02-13 14:37:44 +09:00
aa0552aaa7 하이퍼 파라미터 수정 2026-02-13 14:30:45 +09:00
5d0aca14a6 사용가능 용량 API 수정 2026-02-13 14:19:09 +09:00
af8d59ddfa 이어하기 수정 2026-02-13 14:08:35 +09:00
4f24e09c57 이어하기 수정 2026-02-13 14:04:33 +09:00
4da477706f 이어하기 수정 2026-02-13 13:57:01 +09:00
a070566048 이어하기 로그 수정 2026-02-13 13:26:54 +09:00
a5b3ae613f 트랜젝션처리 임시폴더 uid업데이트 2026-02-13 13:23:53 +09:00
979af088be 트랜젝션처리 임시폴더 uid업데이트 2026-02-13 13:17:27 +09:00
e5a1cab36b Merge branch 'feat/training_260202' of https://kamco.git.gs.dabeeo.com/MVPTeam/kamco-train-api into feat/training_260202 2026-02-13 12:53:04 +09:00
bb67996742 text 2026-02-13 12:53:00 +09:00
1981d6d1ce Merge remote-tracking branch 'origin/feat/training_260202' into feat/training_260202 2026-02-13 12:53:00 +09:00
47f4ffd4db 트랜젝션처리 임시폴더 uid업데이트 2026-02-13 12:52:11 +09:00
195856b846 flush 추가해보기 2026-02-13 12:46:54 +09:00
124da48e51 트랜젝션처리 임시폴더 uid업데이트 2026-02-13 12:38:55 +09:00
02724e9508 주석한거 원복 2026-02-13 12:24:50 +09:00
7ed91ccab9 Merge branch 'feat/training_260202' of https://kamco.git.gs.dabeeo.com/MVPTeam/kamco-train-api into feat/training_260202 2026-02-13 12:23:12 +09:00
a7c13b985d responsePath 셋팅 삭제 2026-02-13 12:23:01 +09:00
352a28b87f 트랜젝션처리 임시폴더 uid업데이트 2026-02-13 12:20:48 +09:00
bf8515163c 주석 처리 2026-02-13 12:10:08 +09:00
2691f6ce16 트랜젝션처리 임시폴더 uid업데이트 2026-02-13 12:00:42 +09:00
7e5aa5e713 트랜젝션처리 임시폴더 uid업데이트 2026-02-13 11:57:48 +09:00
060a815e1c 트랜젝션처리 임시폴더 uid업데이트 2026-02-13 11:55:35 +09:00
1eb4d04779 Merge branch 'feat/training_260202' of https://kamco.git.gs.dabeeo.com/MVPTeam/kamco-train-api into feat/training_260202 2026-02-13 10:50:35 +09:00
f30c0c6d45 다운로드 시 Access-Control-Expose-Headers 추가 2026-02-13 10:50:28 +09:00
12994aab60 파일 count 기능 추가 2026-02-13 10:44:49 +09:00
11d3afe295 파일 count 기능 추가 2026-02-13 10:38:24 +09:00
1e62a8b097 학습실행 step1 할 때 best epoch 업데이트 2026-02-13 10:15:04 +09:00
26a4623aa8 학습데이터 목록 파일 단위 MB 나오게 하기 2026-02-13 09:42:36 +09:00
ce6e4f5aea tmp 파일 링크 수정 2026-02-13 09:10:45 +09:00
c2215836c0 tmp 파일 링크 수정 2026-02-13 08:44:17 +09:00
8c19c996f7 tmp 파일 링크 수정 2026-02-13 08:33:36 +09:00
b5ce3ab1fb 이어하기 수정 2026-02-12 23:01:56 +09:00
e1ceb769dd 학습데이터 다운로드 파일 정보 API 추가 2026-02-12 22:47:13 +09:00
4219b88fb3 학습데이터 다운로드 API 추가 2026-02-12 22:25:55 +09:00
4f94c99b64 이어하기 수정 2026-02-12 22:09:54 +09:00
d42e1afbd4 스케줄러 api 수동 호출 2026-02-12 21:51:53 +09:00
b3b8016673 csv 결과 받아오는 것 변경 2026-02-12 21:45:38 +09:00
79e8259f28 파라미터 변경 2026-02-12 21:30:03 +09:00
032c82c2f0 file 경로 넣기 2026-02-12 21:17:10 +09:00
6204a6e5fa Merge remote-tracking branch 'origin/develop' into feat/training_260202
# Conflicts:
#	src/main/resources/application-prod.yml
2026-02-12 21:15:21 +09:00
4d9c9a86b4 패키징 zip파일 만들기 커밋 2026-02-12 21:09:40 +09:00
83204abfe9 Merge pull request '성공시 csv 파일 테이블에 저장 연결' (#74) from feat/training_260202 into develop
Reviewed-on: #74
2026-02-12 21:01:48 +09:00
5b682c1386 성공시 csv 파일 테이블에 저장 연결 2026-02-12 21:01:24 +09:00
452494d44d Merge pull request '테스트 실행 경로 수정' (#73) from feat/training_260202 into develop
Reviewed-on: #73
2026-02-12 20:49:30 +09:00
8ada26448b 테스트 실행 경로 수정 2026-02-12 20:49:14 +09:00
e442f105bc Merge pull request '도커명 변경' (#72) from feat/training_260202 into develop
Reviewed-on: #72
2026-02-12 20:37:00 +09:00
5e0a771848 도커명 변경 2026-02-12 20:36:38 +09:00
b4c2685059 Merge pull request '도커 설정 추가' (#71) from feat/training_260202 into develop
Reviewed-on: #71
2026-02-12 20:25:16 +09:00
e238f3ca88 도커 설정 추가 2026-02-12 20:24:57 +09:00
97b06eb3b3 Merge pull request '임시파일생성 경로 수정' (#70) from feat/training_260202 into develop
Reviewed-on: #70
2026-02-12 20:03:32 +09:00
ad32ca18ca 임시파일생성 경로 수정 2026-02-12 20:03:03 +09:00
98a1283ebe Merge pull request '임시파일생성 경로 수정' (#69) from feat/training_260202 into develop
Reviewed-on: #69
2026-02-12 19:36:06 +09:00
a10fccaae3 임시파일생성 경로 수정 2026-02-12 19:35:47 +09:00
c3c9191d9d Merge pull request 'hyperparam_with_modeltype' (#68) from feat/dean/hyperparam_with_modelType-bug into develop
Reviewed-on: #68
2026-02-12 19:30:29 +09:00
9fd5a15a72 hyperparam_with_modeltype 2026-02-12 19:30:08 +09:00
12f9de7367 hyperparam_with_modeltype 2026-02-12 19:16:24 +09:00
5455da1e96 hyperparam_with_modeltype 2026-02-12 19:16:13 +09:00
9e803661cd Merge pull request 'feat/training_260202' (#67) from feat/training_260202 into develop
Reviewed-on: #67
2026-02-12 19:14:39 +09:00
b0cf9e77ec Merge branch 'develop' of https://kamco.git.gs.dabeeo.com/MVPTeam/kamco-train-api into develop 2026-02-12 19:14:10 +09:00
c92426aefc Merge branch 'feat/training_260202' of https://kamco.git.gs.dabeeo.com/MVPTeam/kamco-train-api into feat/training_260202 2026-02-12 19:14:09 +09:00
d5b2b8ecec hyperparam_with_modeltype 2026-02-12 19:14:01 +09:00
6185a18a7c 모델목록 검색 조건 상태값 변경 2026-02-12 19:13:56 +09:00
49d3e37458 Merge pull request '임시파일생성 경로 수정' (#66) from feat/training_260202 into develop
Reviewed-on: #66
2026-02-12 19:12:37 +09:00
1fb10830b9 임시파일생성 경로 수정 2026-02-12 19:11:51 +09:00
d7766edd24 Merge pull request 'return 형식 수정' (#65) from feat/training_260202 into develop
Reviewed-on: #65
2026-02-12 18:59:37 +09:00
0bc4453c9c hyperparam_with_modeltype 2026-02-12 18:56:32 +09:00
ae0d30e5da return 형식 수정 2026-02-12 18:55:42 +09:00
37d776dd2c Merge pull request 'hyperparam_with_modeltype' (#64) from feat/dean/hyperparam_with_modelType into develop
Reviewed-on: #64
2026-02-12 18:50:32 +09:00
0c34ea7dcb hyperparam_with_modeltype 2026-02-12 18:48:14 +09:00
3106d36431 Merge pull request '업로드 시 같은 uid로 업로드하지 못하게 조건 추가' (#63) from feat/training_260202 into develop
Reviewed-on: #63
2026-02-12 18:44:49 +09:00
ed48f697a4 업로드 시 같은 uid로 업로드하지 못하게 조건 추가 2026-02-12 18:44:04 +09:00
da92b28d97 Merge pull request '임시파일생성 소프트링크에서 하드링크로 변경' (#62) from feat/training_260202 into develop
Reviewed-on: #62
2026-02-12 18:20:30 +09:00
6c865d26fd 임시파일생성 소프트링크에서 하드링크로 변경 2026-02-12 18:18:44 +09:00
e3f00876f1 Merge pull request '문제되는 하이퍼파라미터 주석처리' (#61) from feat/training_260202 into develop
Reviewed-on: #61
2026-02-12 17:53:11 +09:00
16e156b5b4 문제되는 하이퍼파라미터 주석처리 2026-02-12 17:52:42 +09:00
60962bbc75 Merge pull request '학습실행 mount 경로 수정' (#60) from feat/training_260202 into develop
Reviewed-on: #60
2026-02-12 17:44:15 +09:00
6a939118ff 임시폴더생성 api 추가 2026-02-12 17:43:41 +09:00
64d37dcc08 Merge pull request '임시폴더생성 api 추가' (#59) from feat/training_260202 into develop
Reviewed-on: #59
2026-02-12 17:23:53 +09:00
0c0ae16c2b 임시폴더생성 api 추가 2026-02-12 17:23:34 +09:00
a2490f30e6 Merge pull request '임시폴더생성 api 수정' (#58) from feat/training_260202 into develop
Reviewed-on: #58
2026-02-12 17:14:52 +09:00
953f95aed6 임시폴더생성 api 추가 2026-02-12 17:14:26 +09:00
bd04e1f4e8 Merge pull request '임시폴더생성 api 추가' (#57) from feat/training_260202 into develop
Reviewed-on: #57
2026-02-12 17:03:39 +09:00
85633c8bab 임시폴더생성 api 추가 2026-02-12 17:03:21 +09:00
5fc15937c0 Merge pull request 'feat/training_260202' (#56) from feat/training_260202 into develop
Reviewed-on: #56
2026-02-12 17:00:08 +09:00
8b3940b446 Merge remote-tracking branch 'origin/feat/training_260202' into feat/training_260202 2026-02-12 16:59:44 +09:00
201cfefb6b 임시폴더생성 api 추가 2026-02-12 16:59:39 +09:00
9958b0999a csv 읽는 경로 수정하기, 변수명 수정 2026-02-12 16:58:28 +09:00
3547c28361 Merge pull request 'feat/training_260202' (#55) from feat/training_260202 into develop
Reviewed-on: #55
2026-02-12 16:56:23 +09:00
6c70bfed18 Merge remote-tracking branch 'origin/feat/training_260202' into feat/training_260202
# Conflicts:
#	src/main/java/com/kamco/cd/training/postgres/core/ModelTrainMngCoreService.java
2026-02-12 16:55:52 +09:00
95a75e63f4 임시폴더생성 api 추가 2026-02-12 16:55:10 +09:00
2a1dbee290 Merge pull request '모델학습 1단계 실행중인 것이 있는지 count API' (#54) from feat/training_260202 into develop
Reviewed-on: #54
2026-02-12 16:51:09 +09:00
384a321bf3 모델학습 1단계 실행중인 것이 있는지 count API 2026-02-12 16:50:40 +09:00
f4e97d389b Merge pull request 'file 확인 API 수정' (#53) from feat/training_260202 into develop
Reviewed-on: #53
2026-02-12 16:42:20 +09:00
590810ff0a file 확인 API 수정 2026-02-12 16:41:40 +09:00
a01c872982 Merge pull request 'feat/training_260202' (#52) from feat/training_260202 into develop
Reviewed-on: #52
2026-02-12 16:15:11 +09:00
905a245070 Merge branch 'feat/training_260202' of https://kamco.git.gs.dabeeo.com/MVPTeam/kamco-train-api into feat/training_260202 2026-02-12 16:14:45 +09:00
860ce35a8f docker mount 경로 추가 2026-02-12 16:14:19 +09:00
7f3f5dca40 Merge pull request 'feat/training_260202' (#51) from feat/training_260202 into develop
Reviewed-on: #51
2026-02-12 16:13:19 +09:00
4a0a4e35ed 학습 실행 수정 2026-02-12 16:12:58 +09:00
ae055dca1e 모델등록 수정 2026-02-12 16:01:14 +09:00
26e8e1492f Merge pull request 'feat/training_260202' (#50) from feat/training_260202 into develop
Reviewed-on: #50
2026-02-12 15:52:09 +09:00
8fa722011c 모델등록 수정 2026-02-12 15:51:54 +09:00
17d47d6200 Merge branch 'feat/training_260202' of https://kamco.git.gs.dabeeo.com/MVPTeam/kamco-train-api into feat/training_260202 2026-02-12 15:47:10 +09:00
e178f58fe2 chunk save log 추가 2026-02-12 15:47:06 +09:00
cd0cf5726d Merge pull request 'feat/training_260202' (#49) from feat/training_260202 into develop
Reviewed-on: #49
2026-02-12 15:44:11 +09:00
8e4bea53da 모델등록 수정 2026-02-12 15:43:52 +09:00
7a22d8ba73 containerName 생성 변경 2026-02-12 15:39:12 +09:00
2df4a7a80b csv 파일 읽는 경로 읽어서 수정, train은 epoch + 1 해서 저장 2026-02-12 15:24:30 +09:00
b451f697bc 모델 마스터 테이블 request,response 경로 추가 2026-02-12 14:59:35 +09:00
7e9c867f34 Merge pull request '모델 등록할 때 step1State를 READY로 업데이트' (#48) from feat/training_260202 into develop
Reviewed-on: #48
2026-02-12 14:35:52 +09:00
130e85f8a1 모델 등록할 때 step1State를 READY로 업데이트 2026-02-12 14:35:17 +09:00
9e713cb49d Merge pull request '업로드 로직 재수정' (#47) from feat/training_260202 into develop
Reviewed-on: #47
2026-02-12 14:21:57 +09:00
51dfa97900 업로드 로직 재수정 2026-02-12 14:21:08 +09:00
87c6b599b4 Merge pull request 'feat/training_260202' (#46) from feat/training_260202 into develop
Reviewed-on: #46
2026-02-12 12:10:04 +09:00
f50855a822 Merge branch 'feat/training_260202' of https://kamco.git.gs.dabeeo.com/MVPTeam/kamco-train-api into feat/training_260202 2026-02-12 12:08:04 +09:00
8d416317a8 베스트 에폭 API, 2단계 실행 시 best epoch 업데이트 2026-02-12 12:07:44 +09:00
22aa071476 Merge pull request 'feat/training_260202' (#45) from feat/training_260202 into develop
Reviewed-on: #45
2026-02-12 12:06:04 +09:00
a83bd09f8f containerName 생성 변경 2026-02-12 12:05:30 +09:00
96035f864a containerName 생성 변경 2026-02-12 11:42:38 +09:00
fd7dfd7e7f containerName 생성 변경 2026-02-12 11:10:28 +09:00
190b93bee8 실행 오류 수정 2026-02-12 10:58:51 +09:00
c5f19cc961 실행 오류 수정 2026-02-12 10:58:32 +09:00
c56c0ca605 실행 오류 수정 2026-02-12 10:58:26 +09:00
c6e721aa37 실행 오류 수정 2026-02-12 10:58:12 +09:00
6572e17f00 실행 오류 수정 2026-02-12 10:51:15 +09:00
be6365807c Merge pull request '실행 오류 수정' (#43) from feat/training_260202 into develop
Reviewed-on: #43
2026-02-12 10:20:05 +09:00
d2fff7dfde 실행 오류 수정 2026-02-12 10:19:44 +09:00
f66bc22c95 Merge pull request '실행 오류 수정' (#42) from feat/training_260202 into develop
Reviewed-on: #42
2026-02-12 10:14:54 +09:00
3367d0e7be 실행 오류 수정 2026-02-12 10:14:32 +09:00
352ec6ccb0 Merge pull request 'feat/training_260202' (#41) from feat/training_260202 into develop
Reviewed-on: #41
2026-02-12 09:53:02 +09:00
6a989255a3 모델별 데이터셋 목록 - G2,G3 dataTypeName 추가 2026-02-12 09:52:24 +09:00
878b21573f 테스트 실행 추가 2026-02-11 22:00:35 +09:00
0602db1436 Merge pull request '테스트 실행 추가' (#40) from feat/training_260202 into develop
Reviewed-on: #40
2026-02-11 21:58:58 +09:00
2f8bd1f98c 테스트 실행 추가 2026-02-11 21:58:25 +09:00
75231ccbba Merge pull request '추론 실행 추가' (#39) from feat/training_260202 into develop
Reviewed-on: #39
2026-02-11 20:22:01 +09:00
1249a80da5 추론 실행 추가 2026-02-11 20:21:25 +09:00
00c78eb42f Merge pull request '성능정보 그래프 데이터 API 추가' (#38) from feat/training_260202 into develop
Reviewed-on: #38
2026-02-11 19:52:23 +09:00
35767adba1 성능정보 그래프 데이터 API 추가 2026-02-11 19:52:00 +09:00
47a2a159ef Merge pull request 'test metrics 스케줄 추가' (#37) from feat/training_260202 into develop
Reviewed-on: #37
2026-02-11 19:10:37 +09:00
95548223cd test metrics 스케줄 추가 2026-02-11 19:09:58 +09:00
2debdc5312 Merge pull request 'feat/training_260202' (#36) from feat/training_260202 into develop
Reviewed-on: #36
2026-02-11 18:51:01 +09:00
207cc47f1b 스케줄 주석 2026-02-11 18:50:43 +09:00
b6338bce8e 테이블 구조 변경 2026-02-11 18:49:59 +09:00
2cfa2adcf5 tb_model_master 컬럼 추가 2026-02-11 17:21:48 +09:00
d7e19abfc9 uploadRate 로직 수정 2026-02-11 17:06:02 +09:00
c843703ee7 Merge pull request 'file 가져오기 86 호출하는 거로 추가' (#35) from feat/training_260202 into develop
Reviewed-on: #35
2026-02-11 16:53:25 +09:00
133ea6b1ba file 가져오기 86 호출하는 거로 추가 2026-02-11 16:49:48 +09:00
0df977ae81 Merge pull request '업로드 로직 86으로 수행하기 수정' (#34) from feat/training_260202 into develop
Reviewed-on: #34
2026-02-11 16:33:03 +09:00
3e39006822 업로드 로직 86으로 수행하기 수정 2026-02-11 16:32:40 +09:00
3ec1a71406 Merge pull request '업로드 로직 수정' (#33) from feat/training_260202 into develop
Reviewed-on: #33
2026-02-11 15:53:21 +09:00
16009f1623 업로드 로직 수정 2026-02-11 15:52:57 +09:00
41911014c9 Merge pull request '업로드 로직 수정' (#32) from feat/training_260202 into develop
Reviewed-on: #32
2026-02-11 15:44:54 +09:00
8ea32ce675 업로드 로직 수정 2026-02-11 15:44:18 +09:00
a4ac80c787 Merge pull request '업로드 경로 수정' (#31) from feat/training_260202 into develop
Reviewed-on: #31
2026-02-11 15:11:02 +09:00
3a5d136d34 업로드 경로 수정 2026-02-11 15:10:37 +09:00
2f63b9ddcd Merge pull request 'feat/training_260202' (#30) from feat/training_260202 into develop
Reviewed-on: #30
2026-02-11 14:08:58 +09:00
92de48b55e 전이학습 상세 로직 수정 2026-02-11 14:08:21 +09:00
224ddae68b 전이학습 상세 수정 2026-02-11 14:05:15 +09:00
885b72a0c6 Merge pull request '모델별 데이터셋 목록 조회 수정' (#29) from feat/training_260202 into develop
Reviewed-on: #29
2026-02-11 12:29:08 +09:00
9ac00d37c5 모델별 데이터셋 목록 조회 수정 2026-02-11 12:28:38 +09:00
fbb5a34867 Merge pull request '업로드 경로 원복' (#28) from feat/training_260202 into develop
Reviewed-on: #28
2026-02-11 12:12:43 +09:00
e25fc01b25 업로드 경로 원복 2026-02-11 12:12:08 +09:00
6b3f22dd66 Merge pull request '업로드 파일 max 수정' (#27) from feat/training_260202 into develop
Reviewed-on: #27
2026-02-11 11:53:04 +09:00
abc2c8e806 업로드 파일 max 수정 2026-02-11 11:52:41 +09:00
29e1d0ec7e Merge pull request '업로드 경로 수정' (#26) from feat/training_260202 into develop
Reviewed-on: #26
2026-02-11 11:41:49 +09:00
8d379d064c 업로드 경로 수정 2026-02-11 11:38:55 +09:00
a2072e0148 Merge pull request 'feat/training_260202' (#25) from feat/training_260202 into develop
Reviewed-on: #25
2026-02-11 10:28:38 +09:00
6352f74b08 업로드 성공 후 COMPLETED 해주기 2026-02-10 18:28:42 +09:00
025b573859 학습데이터 obj-list API geojson 로직 수정 2026-02-10 16:36:07 +09:00
0e9fa80092 학습데이터 등록 로직 커밋 2026-02-10 16:15:21 +09:00
2d5de88a6b Merge pull request '학습데이터 업로드, unzip 로직 진행중' (#24) from feat/training_260202 into develop
Reviewed-on: #24
2026-02-10 10:44:28 +09:00
89744d2aa1 학습데이터 업로드, unzip 로직 진행중 2026-02-10 10:43:40 +09:00
eda1d19942 Merge pull request '스웨거 로그인 설정 수정' (#23) from feat/training_260202 into develop
Reviewed-on: #23
2026-02-06 14:54:56 +09:00
b4a4486560 스웨거 로그인 설정 수정 2026-02-06 14:51:45 +09:00
653717a074 Merge pull request 'feat/training_260202' (#22) from feat/training_260202 into develop
Reviewed-on: #22
2026-02-06 11:09:36 +09:00
7cc3392856 사용 가능 공간 조회 API 추가 2026-02-06 11:09:13 +09:00
def84d2b1c 학습데이터 obj 삭제 스웨거 문구 수정 2026-02-06 10:33:12 +09:00
679795d14d Merge pull request '전이학습 추가' (#21) from feat/training_260202 into develop
Reviewed-on: #21
2026-02-05 18:23:30 +09:00
0a7f01a2f5 전이학습 추가 2026-02-05 18:23:07 +09:00
9655c62d35 Merge pull request '하이퍼 파라미터 최적값 조회 수정' (#20) from feat/training_260202 into develop
Reviewed-on: #20
2026-02-05 15:22:56 +09:00
db6844f0e7 하이퍼 파라미터 최적값 조회 수정 2026-02-05 15:22:34 +09:00
af16933378 Merge pull request 'feat/training_260202' (#19) from feat/training_260202 into develop
Reviewed-on: #19
2026-02-05 15:08:41 +09:00
29b653a4e9 데이터셋 상세조회 class 조회 추가 2026-02-05 15:08:22 +09:00
693e3ef3ab 모델학습 데이터셋 선택 목록 수정 2026-02-05 14:22:37 +09:00
03135a972a Merge pull request '모델학습 데이터셋 선택 목록 수정' (#18) from feat/training_260202 into develop
Reviewed-on: #18
2026-02-05 13:59:00 +09:00
381b7d7e0b 모델학습 데이터셋 선택 목록 수정 2026-02-05 13:58:31 +09:00
947cba2742 Merge pull request 'feat/training_260202' (#17) from feat/training_260202 into develop
Reviewed-on: #17
2026-02-04 19:54:23 +09:00
f038fdd1db 모델 상세 추가 API 커밋 2026-02-04 19:54:00 +09:00
474a3c119e 모델학습 config 정보 조회 추가 2026-02-04 19:51:43 +09:00
b2be43a76e 모델 상세 API 커밋 2026-02-04 19:46:57 +09:00
ce69bacb01 모델학습 데이터 등록 수정 2026-02-04 19:15:10 +09:00
200b384e19 모델 종류 이름 변경 2026-02-04 18:41:34 +09:00
350d622e5a 미사용 소스 정리 2026-02-04 18:26:02 +09:00
b25fc6fe68 Merge pull request 'feat/training_260202' (#16) from feat/training_260202 into develop
Reviewed-on: #16
2026-02-04 18:01:15 +09:00
6cdf4efda6 데이터셋 등록 추가 2026-02-04 18:00:56 +09:00
7d866e5869 데이터셋 테이블 수정 2026-02-04 15:13:13 +09:00
5bc59c0e0b Merge pull request '데이터셋 조회 수정' (#15) from feat/training_260202 into develop
Reviewed-on: #15
2026-02-04 14:11:00 +09:00
3c0a12da4e 데이터셋 조회 수정 2026-02-04 14:10:46 +09:00
cdac9d6148 Merge pull request '모델학습 설정 dto 수정' (#14) from feat/training_260202 into develop
Reviewed-on: #14
2026-02-04 14:03:41 +09:00
abe4272227 모델학습 설정 dto 수정 2026-02-04 14:03:25 +09:00
d238debcc8 Merge pull request '모델학습 설정 dto 수정' (#13) from feat/training_260202 into develop
Reviewed-on: #13
2026-02-04 13:57:00 +09:00
fdfda049f8 모델학습 설정 dto 수정 2026-02-04 13:56:32 +09:00
3e780ef007 Merge pull request 'feat/training_260202' (#12) from feat/training_260202 into develop
Reviewed-on: #12
2026-02-04 12:32:36 +09:00
2c825b14ee 모델학습 설정 dto 수정 2026-02-04 12:32:22 +09:00
f9d081970d 모델학습 설정 dto 수정 2026-02-04 12:29:09 +09:00
2110f395b7 Merge pull request 'dataset 테이블 수정, 모델학습 설정 dto 추가' (#11) from feat/training_260202 into develop
Reviewed-on: #11
2026-02-04 12:25:19 +09:00
60d45ee2ce dataset 테이블 수정, 모델학습 설정 dto 추가 2026-02-04 12:24:55 +09:00
50464c1aa8 Merge pull request 'feat/training_260202' (#10) from feat/training_260202 into develop
Reviewed-on: #10
2026-02-03 18:52:24 +09:00
f1ad59d0b1 데이터셋 API 커밋 2026-02-03 18:51:56 +09:00
19644e5c9f dataset 테이블 수정 2026-02-03 18:51:35 +09:00
4c80017fc5 dataset 테이블 수정 2026-02-03 18:46:00 +09:00
5dfcd3d181 Merge pull request '하이퍼파라미터 , 모델관리 수정' (#9) from feat/training_260202 into develop
Reviewed-on: #9
2026-02-03 18:25:13 +09:00
6e99c209d6 하이퍼파라미터 , 모델관리 수정 2026-02-03 18:24:49 +09:00
38b037da31 Merge pull request 'feat/training_260202' (#8) from feat/training_260202 into develop
Reviewed-on: #8
2026-02-03 16:32:02 +09:00
d66711e4f4 하이퍼파라미터 기능 추가 2026-02-03 16:31:43 +09:00
44878e9c37 하이퍼파라미터 기능 추가 2026-02-03 15:15:25 +09:00
139 changed files with 11461 additions and 2185 deletions

View File

@@ -1,6 +1,11 @@
# Stage 1: Build stage (gradle build는 Jenkins에서 이미 수행)
FROM eclipse-temurin:21-jre-jammy
# docker CLI 설치 (컨테이너에서 호스트 Docker 제어용) 260212 추가
RUN apt-get update && \
apt-get install -y --no-install-recommends docker.io ca-certificates && \
rm -rf /var/lib/apt/lists/*
# 작업 디렉토리 설정
WORKDIR /app

View File

@@ -3,6 +3,7 @@ plugins {
id 'org.springframework.boot' version '3.5.7'
id 'io.spring.dependency-management' version '1.1.7'
id 'com.diffplug.spotless' version '6.25.0'
id 'idea'
}
group = 'com.kamco.cd'
@@ -21,11 +22,23 @@ configurations {
}
}
// QueryDSL 생성된 소스 디렉토리 정의
def generatedSourcesDir = file("$buildDir/generated/sources/annotationProcessor/java/main")
repositories {
mavenCentral()
maven { url "https://repo.osgeo.org/repository/release/" }
}
// Gradle이 생성된 소스를 컴파일 경로에 포함하도록 설정
sourceSets {
main {
java {
srcDirs += generatedSourcesDir
}
}
}
dependencies {
implementation 'org.springframework.boot:spring-boot-starter-data-jpa'
implementation 'org.springframework.boot:spring-boot-starter-web'
@@ -83,6 +96,23 @@ dependencies {
implementation 'io.hypersistence:hypersistence-utils-hibernate-63:3.7.0'
implementation 'org.reflections:reflections:0.10.2'
implementation 'com.jcraft:jsch:0.1.55'
implementation 'org.apache.commons:commons-csv:1.10.0'
}
// IntelliJ가 생성된 소스를 인식하도록 설정
idea {
module {
// 소스 디렉토리로 인식
sourceDirs += generatedSourcesDir
// Generated Sources Root로 마킹 (IntelliJ에서 특별 처리)
generatedSourceDirs += generatedSourcesDir
// 소스 및 Javadoc 다운로드
downloadJavadoc = true
downloadSources = true
}
}
configurations.configureEach {
@@ -93,6 +123,21 @@ tasks.named('test') {
useJUnitPlatform()
}
// 컴파일 전 생성된 소스 디렉토리 생성 보장
tasks.named('compileJava') {
doFirst {
generatedSourcesDir.mkdirs()
}
}
// 생성된 소스 정리 태스크
tasks.register('cleanGeneratedSources', Delete) {
delete generatedSourcesDir
}
tasks.named('clean') {
dependsOn 'cleanGeneratedSources'
}
bootJar {
archiveFileName = 'ROOT.jar'

View File

@@ -14,6 +14,8 @@ services:
- /mnt/nfs_share/images:/app/original-images
- /mnt/nfs_share/model_output:/app/model-outputs
- /mnt/nfs_share/train_dataset:/app/train-dataset
- /home/kcomu/data:/home/kcomu/data
- /var/run/docker.sock:/var/run/docker.sock
networks:
- kamco-cds
restart: unless-stopped

View File

@@ -23,7 +23,8 @@ public class JwtAuthenticationFilter extends OncePerRequestFilter {
private final UserDetailsService userDetailsService;
private static final AntPathMatcher PATH_MATCHER = new AntPathMatcher();
private static final String[] EXCLUDE_PATHS = {
"/api/auth/signin", "/api/auth/refresh", "/api/auth/logout", "/api/members/*/password"
// "/api/auth/signin", "/api/auth/refresh", "/api/auth/logout", "/api/members/*/password"
"/api/auth/signin", "/api/auth/refresh", "/api/auth/logout"
};
@Override

View File

@@ -20,7 +20,7 @@ import org.springframework.web.bind.annotation.*;
@Tag(name = "공통코드 관리", description = "공통코드 관리 API")
@RestController
@RequiredArgsConstructor
@RequestMapping("/api/code")
@RequestMapping("/api/common-code")
public class CommonCodeApiController {
private final CommonCodeService commonCodeService;

View File

@@ -0,0 +1,48 @@
package com.kamco.cd.training.common.download;
import com.kamco.cd.training.common.download.dto.DownloadSpec;
import com.kamco.cd.training.common.utils.UserUtil;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import lombok.RequiredArgsConstructor;
import org.springframework.http.HttpHeaders;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Service;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
@Service
@RequiredArgsConstructor
public class DownloadExecutor {
private final UserUtil userUtil;
public ResponseEntity<StreamingResponseBody> stream(DownloadSpec spec) throws IOException {
if (!Files.isReadable(spec.filePath())) {
return ResponseEntity.notFound().build();
}
StreamingResponseBody body =
os -> {
try (InputStream in = Files.newInputStream(spec.filePath())) {
in.transferTo(os);
os.flush();
} catch (Exception e) {
// 고용량은 중간 끊김 흔하니까 throw 금지
}
};
String fileName =
spec.downloadName() != null
? spec.downloadName()
: spec.filePath().getFileName().toString();
return ResponseEntity.ok()
.contentType(
spec.contentType() != null ? spec.contentType() : MediaType.APPLICATION_OCTET_STREAM)
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + fileName + "\"")
.body(body);
}
}

View File

@@ -0,0 +1,19 @@
package com.kamco.cd.training.common.download;
import org.springframework.util.AntPathMatcher;
public final class DownloadPaths {
private DownloadPaths() {}
public static final String[] PATTERNS = {
"/api/inference/download/**", "/api/training-data/stage/download/**"
};
public static boolean matches(String uri) {
AntPathMatcher m = new AntPathMatcher();
for (String p : PATTERNS) {
if (m.match(p, uri)) return true;
}
return false;
}
}

View File

@@ -0,0 +1,81 @@
package com.kamco.cd.training.common.download;
import jakarta.servlet.http.HttpServletRequest;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;
import org.springframework.core.io.FileSystemResource;
import org.springframework.core.io.Resource;
import org.springframework.core.io.support.ResourceRegion;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpRange;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Component;
@Component
public class RangeDownloadResponder {
public ResponseEntity<?> buildZipResponse(
Path filePath, String downloadFileName, HttpServletRequest request) throws IOException {
if (!Files.isRegularFile(filePath)) {
return ResponseEntity.notFound().build();
}
long totalSize = Files.size(filePath);
Resource resource = new FileSystemResource(filePath);
String disposition = "attachment; filename=\"" + downloadFileName + "\"";
String rangeHeader = request.getHeader(HttpHeaders.RANGE);
// 🔥 공통 헤더 (여기 고정)
ResponseEntity.BodyBuilder base =
ResponseEntity.ok()
.contentType(MediaType.APPLICATION_OCTET_STREAM)
.header(HttpHeaders.CONTENT_DISPOSITION, disposition)
.header(HttpHeaders.ACCEPT_RANGES, "bytes")
.header("Access-Control-Expose-Headers", "Content-Disposition")
.header("X-Accel-Buffering", "no");
if (rangeHeader == null || rangeHeader.isBlank()) {
return base.contentLength(totalSize).body(resource);
}
List<HttpRange> ranges;
try {
ranges = HttpRange.parseRanges(rangeHeader);
} catch (IllegalArgumentException ex) {
return ResponseEntity.status(416)
.header(HttpHeaders.CONTENT_RANGE, "bytes */" + totalSize)
.header("X-Accel-Buffering", "no")
.build();
}
HttpRange range = ranges.get(0);
long start = range.getRangeStart(totalSize);
long end = range.getRangeEnd(totalSize);
if (start >= totalSize) {
return ResponseEntity.status(416)
.header(HttpHeaders.CONTENT_RANGE, "bytes */" + totalSize)
.header("X-Accel-Buffering", "no")
.build();
}
long regionLength = end - start + 1;
ResourceRegion region = new ResourceRegion(resource, start, regionLength);
return ResponseEntity.status(206)
.contentType(MediaType.APPLICATION_OCTET_STREAM)
.header(HttpHeaders.CONTENT_DISPOSITION, disposition)
.header(HttpHeaders.ACCEPT_RANGES, "bytes")
.header("Access-Control-Expose-Headers", "Content-Disposition")
.header("X-Accel-Buffering", "no")
.header(HttpHeaders.CONTENT_RANGE, "bytes " + start + "-" + end + "/" + totalSize)
.contentLength(regionLength)
.body(region);
}
}

View File

@@ -0,0 +1,12 @@
package com.kamco.cd.training.common.download.dto;
import java.nio.file.Path;
import java.util.UUID;
import org.springframework.http.MediaType;
public record DownloadSpec(
UUID uuid, // 다운로드 식별(로그/정책용)
Path filePath, // 실제 파일 경로
String downloadName, // 사용자에게 보일 파일명
MediaType contentType // 보통 OCTET_STREAM
) {}

View File

@@ -0,0 +1,160 @@
package com.kamco.cd.training.common.dto;
import com.kamco.cd.training.common.enums.ModelType;
import io.swagger.v3.oas.annotations.media.Schema;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
@Getter
@Setter
@AllArgsConstructor
@NoArgsConstructor
public class HyperParam {
@Schema(description = "모델", example = "G1")
private ModelType model; // G1, G2, G3
// -------------------------
// Important
// -------------------------
@Schema(description = "백본 네트워크", example = "large")
private String backbone; // backbone
@Schema(description = "입력 이미지 크기(H,W)", example = "512,512")
private String inputSize; // input_size
@Schema(description = "크롭 크기(H,W 또는 단일값)", example = "256,256")
private String cropSize; // crop_size
@Schema(description = "배치 크기(Per GPU)", example = "16")
private Integer batchSize; // batch_size
// -------------------------
// Data
// -------------------------
@Schema(description = "Train dataloader workers", example = "16")
private Integer trainNumWorkers; // train_num_workers
@Schema(description = "Val dataloader workers", example = "8")
private Integer valNumWorkers; // val_num_workers
@Schema(description = "Test dataloader workers", example = "8")
private Integer testNumWorkers; // test_num_workers
@Schema(description = "Train shuffle 여부", example = "true")
private Boolean trainShuffle; // train_shuffle
@Schema(description = "Train persistent workers 여부", example = "true")
private Boolean trainPersistent; // train_persistent
@Schema(description = "Val persistent workers 여부", example = "true")
private Boolean valPersistent; // val_persistent
// -------------------------
// Model Architecture
// -------------------------
@Schema(description = "Drop Path 비율", example = "0.3")
private Double dropPathRate; // drop_path_rate
@Schema(description = "Freeze 단계(-1:None)", example = "-1")
private Integer frozenStages; // frozen_stages
@Schema(description = "Neck 결합 정책", example = "abs_diff")
private String neckPolicy; // neck_policy
@Schema(description = "디코더 채널 구성", example = "512,256,128,64")
private String decoderChannels; // decoder_channels
@Schema(description = "클래스별 가중치", example = "1,10")
private String classWeight; // class_weight
// -------------------------
// Loss & Optimization
// -------------------------
@Schema(description = "학습률", example = "0.00006")
private Double learningRate; // learning_rate
@Schema(description = "Weight Decay", example = "0.05")
private Double weightDecay; // weight_decay
@Schema(description = "Layer Decay Rate", example = "0.9")
private Double layerDecayRate; // layer_decay_rate
@Schema(description = "DDP unused params 탐색 여부", example = "true")
private Boolean ddpFindUnusedParams; // ddp_find_unused_params
@Schema(description = "Loss 계산 제외 인덱스", example = "255")
private Integer ignoreIndex; // ignore_index
@Schema(description = "레이어 깊이", example = "24")
private Integer numLayers; // num_layers
// -------------------------
// Evaluation
// -------------------------
@Schema(description = "평가 지표 목록", example = "mFscore,mIoU")
private String metrics; // metrics
@Schema(description = "Best 모델 선정 기준 지표", example = "changed_fscore")
private String saveBest; // save_best
@Schema(description = "Best 모델 선정 규칙", example = "less")
private String saveBestRule; // save_best_rule
@Schema(description = "검증 수행 주기(Epoch)", example = "10")
private Integer valInterval; // val_interval
@Schema(description = "로그 기록 주기(Iteration)", example = "400")
private Integer logInterval; // log_interval
@Schema(description = "시각화 저장 주기(Epoch)", example = "1")
private Integer visInterval; // vis_interval
// -------------------------
// Augmentation
// -------------------------
@Schema(description = "회전 적용 확률", example = "0.5")
private Double rotProb; // rot_prob
@Schema(description = "회전 각도 범위(Min,Max)", example = "-20,20")
private String rotDegree; // rot_degree
@Schema(description = "반전 적용 확률", example = "0.5")
private Double flipProb; // flip_prob
@Schema(description = "채널 교환 확률", example = "0.5")
private Double exchangeProb; // exchange_prob
@Schema(description = "밝기 변화량", example = "10")
private Integer brightnessDelta; // brightness_delta
@Schema(description = "대비 범위(Min,Max)", example = "0.8,1.2")
private String contrastRange; // contrast_range
@Schema(description = "채도 범위(Min,Max)", example = "0.8,1.2")
private String saturationRange; // saturation_range
@Schema(description = "색조 변화량", example = "10")
private Integer hueDelta; // hue_delta
// -------------------------
// Hardware
// -------------------------
@Schema(description = "사용 GPU 개수", example = "4")
private Integer gpuCnt; // gpu_cnt
@Schema(description = "사용 GPU ID 목록", example = "0,1,2,3")
private String gpuIds; // gpu_ids
@Schema(description = "분산학습 마스터 포트", example = "1122")
private Integer masterPort; // master_port
// -------------------------
// Memo
// -------------------------
@Schema(description = "메모", example = "하이퍼파라미터 신규등록")
private String memo; // memo
}

View File

@@ -6,21 +6,22 @@ import lombok.Getter;
@Getter
@AllArgsConstructor
public enum DetectionClassification {
BUILDING("building", "건물", 10),
CONTAINER("container", "컨테이너", 20),
FIELD("field", "경작지", 30),
FOREST("forest", "", 40),
GRASS("grass", "초지", 50),
GREENHOUSE("greenhouse", "비닐하우스", 60),
LAND("land", "일반토지", 70),
ORCHARD("orchard", "과수원", 80),
ROAD("road", "도로", 90),
STONE("stone", "모래/자갈", 100),
TANK("tank", "물탱크", 110),
TUMULUS("tumulus", "토분(무덤)", 120),
WASTE("waste", "폐기물", 130),
WATER("water", "", 140),
ETC("ETC", "기타", 200); // For 'etc' (miscellaneous/other)
ROAD("road", "도로", 10),
BUILDING("building", "건물", 20),
GREENHOUSE("greenhouse", "비닐하우스", 30),
FIELD("field", "논/밭", 40),
ORCHARD("orchard", "과수원", 50),
GRASS("grass", "초지", 60),
FOREST("forest", "", 70),
WATER("water", "", 80),
STONE("stone", "모래/자갈", 90),
WASTE("waste", "적치물", 100),
CONTAINER("container", "컨테이너", 110),
LAND("land", "일반토지", 120),
SOLAR("solar", "태양광", 130),
TANK("tank", "탱크", 140),
NDC("NDC", "미분류", 150),
ETC("ETC", "기타", 160);
private final String id;
private final String desc;

View File

@@ -0,0 +1,27 @@
package com.kamco.cd.training.common.enums;
import com.kamco.cd.training.common.utils.enums.CodeExpose;
import com.kamco.cd.training.common.utils.enums.EnumType;
import lombok.AllArgsConstructor;
import lombok.Getter;
@CodeExpose
@Getter
@AllArgsConstructor
public enum HyperParamSelectType implements EnumType {
OPTIMIZED("최적화 파라미터"),
EXISTING("기존 파라미터"),
NEW("신규 파라미터");
private final String desc;
@Override
public String getId() {
return name();
}
@Override
public String getText() {
return desc;
}
}

View File

@@ -0,0 +1,27 @@
package com.kamco.cd.training.common.enums;
import com.kamco.cd.training.common.utils.enums.EnumType;
import lombok.AllArgsConstructor;
import lombok.Getter;
@Getter
@AllArgsConstructor
public enum JobStatusType implements EnumType {
QUEUED("대기중"),
RUNNING("실행중"),
SUCCESS("성공"),
FAILED("실패"),
CANCELED("취소");
private final String desc;
@Override
public String getId() {
return name();
}
@Override
public String getText() {
return desc;
}
}

View File

@@ -0,0 +1,24 @@
package com.kamco.cd.training.common.enums;
import com.kamco.cd.training.common.utils.enums.EnumType;
import lombok.AllArgsConstructor;
import lombok.Getter;
@Getter
@AllArgsConstructor
public enum JobType implements EnumType {
TRAIN("학습"),
TEST("테스트");
private final String desc;
@Override
public String getId() {
return name();
}
@Override
public String getText() {
return desc;
}
}

View File

@@ -0,0 +1,35 @@
package com.kamco.cd.training.common.enums;
import com.kamco.cd.training.common.utils.enums.CodeExpose;
import com.kamco.cd.training.common.utils.enums.EnumType;
import java.util.Arrays;
import lombok.AllArgsConstructor;
import lombok.Getter;
@CodeExpose
@Getter
@AllArgsConstructor
public enum ModelType implements EnumType {
G1("G1"),
G2("G2"),
G3("G3");
private String desc;
public static ModelType getValueData(String modelNo) {
return Arrays.stream(ModelType.values())
.filter(m -> m.getId().equals(modelNo))
.findFirst()
.orElse(G1);
}
@Override
public String getId() {
return name();
}
@Override
public String getText() {
return desc;
}
}

View File

@@ -10,13 +10,21 @@ import lombok.Getter;
@AllArgsConstructor
public enum TrainStatusType implements EnumType {
// @formatter:off
READY("READY", "대기"),
ING("ING", "진행중"),
COMPLETED("COMPLETED", "완료"),
STOPPED("STOPPED", "중단됨"),
ERROR("ERROR", "오류");
// @formatter:on
READY("대기"),
IN_PROGRESS("진행중"),
COMPLETED("완료"),
STOPPED("중단됨"),
ERROR("오류");
private final String id;
private final String text;
private final String desc;
@Override
public String getId() {
return name();
}
@Override
public String getText() {
return desc;
}
}

View File

@@ -0,0 +1,26 @@
package com.kamco.cd.training.common.enums;
import com.kamco.cd.training.common.utils.enums.CodeExpose;
import com.kamco.cd.training.common.utils.enums.EnumType;
import lombok.AllArgsConstructor;
import lombok.Getter;
@CodeExpose
@Getter
@AllArgsConstructor
public enum TrainType implements EnumType {
GENERAL("일반"),
TRANSFER("전이");
private final String desc;
@Override
public String getId() {
return name();
}
@Override
public String getText() {
return desc;
}
}

View File

@@ -0,0 +1,50 @@
package com.kamco.cd.training.common.service;
import java.nio.file.FileStore;
import java.nio.file.Files;
import java.nio.file.Path;
public class FormatStorage {
private FormatStorage() {}
/** 디스크 사용량 조회 */
public static DiskUsage getDiskUsage(Path path) throws Exception {
FileStore store = Files.getFileStore(path);
long total = store.getTotalSpace();
long usable = store.getUsableSpace();
return new DiskUsage(path.toString(), total, usable);
}
/** 디스크 사용량 DTO */
public record DiskUsage(String path, long totalBytes, long usableBytes) {
public long usedBytes() {
return totalBytes - usableBytes;
}
public double usedPercent() {
return totalBytes == 0 ? 0.0 : (usedBytes() * 100.0) / totalBytes;
}
public String totalText() {
return formatStorageSize(totalBytes);
}
public String usableText() {
return formatStorageSize(usableBytes);
}
/** 저장공간을 사람이 읽기 좋은 단위로 변환 (GB / MB) */
private static String formatStorageSize(long bytes) {
double gb = bytes / 1024.0 / 1024 / 1024;
if (gb >= 1) {
return String.format("%.1f GB", gb);
}
double mb = bytes / 1024.0 / 1024;
return String.format("%.0f MB", mb);
}
}
}

View File

@@ -2,13 +2,21 @@ package com.kamco.cd.training.common.utils;
import static java.lang.String.CASE_INSENSITIVE_ORDER;
import com.jcraft.jsch.ChannelExec;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.Session;
import com.kamco.cd.training.common.exception.CustomApiException;
import com.kamco.cd.training.config.api.ApiResponseDto.ApiResponseCode;
import io.swagger.v3.oas.annotations.media.Schema;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.FileReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.UncheckedIOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
@@ -21,17 +29,23 @@ import java.util.Arrays;
import java.util.Comparator;
import java.util.Date;
import java.util.List;
import java.util.Properties;
import java.util.Set;
import java.util.function.Predicate;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import lombok.Getter;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.io.FilenameUtils;
import org.geotools.coverage.grid.GridCoverage2D;
import org.geotools.gce.geotiff.GeoTiffReader;
import org.springframework.http.HttpStatus;
import org.springframework.util.FileSystemUtils;
import org.springframework.web.multipart.MultipartFile;
@Slf4j
public class FIleChecker {
static SimpleDateFormat dttmFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
@@ -492,16 +506,37 @@ public class FIleChecker {
public static boolean multipartChunkSaveTo(
MultipartFile mfile, String targetPath, int chunkIndex) {
File dest = new File(targetPath, String.valueOf(chunkIndex));
boolean fileUpload = true;
try {
File dir = new File(targetPath);
log.info("targetPath={}", targetPath);
log.info("absolute targetPath={}", dir.getAbsolutePath());
if (!dir.exists()) {
dir.mkdirs();
}
File dest = new File(dir, String.valueOf(chunkIndex));
log.info("real save path = {}", dest.getAbsolutePath());
log.info("chunkIndex={}, uploadSize={}", chunkIndex, mfile.getSize());
log.info("savedSize={}", dest.length());
// ⭐ 핵심
if (dest.exists()) {
dest.delete();
}
log.info("after delete={}", dest.length());
mfile.transferTo(dest);
log.info("after transfer size={}", dest.length());
log.info("after transfer exists={}", dest.exists());
return true;
} catch (IOException e) {
log.error("chunk save error", e);
return false;
}
return true;
}
public static boolean deleteFolder(String path) {
@@ -680,4 +715,148 @@ public class FIleChecker {
this.lastModified = lastModified;
}
}
public static void unzip(String fileName, String destDirectory) throws IOException {
String zipFilePath = destDirectory + File.separator + fileName;
log.info("fileName : {}", fileName);
log.info("destDirectory : {}", destDirectory);
log.info("zipFilePath : {}", zipFilePath);
// zip 이름으로 폴더 생성 (확장자 제거)
String folderName =
fileName.endsWith(".zip") ? fileName.substring(0, fileName.length() - 4) : fileName;
log.info("folderName : {}", folderName);
File destDir = new File(destDirectory, folderName);
log.info("destDir : {}", destDir);
// 동일 폴더가 이미 있으면 삭제
log.info("111 destDir.exists() : {}", destDir.exists());
if (destDir.exists()) {
deleteDirectoryRecursively(destDir.toPath());
}
log.info("222 destDir.exists() : {}", destDir.exists());
if (!destDir.exists()) {
log.info("mkdirs : {}", destDir.exists());
destDir.mkdirs();
}
try (ZipInputStream zis = new ZipInputStream(new FileInputStream(zipFilePath))) {
ZipEntry zipEntry = zis.getNextEntry();
while (zipEntry != null) {
File newFile = newFile(destDir, zipEntry);
if (zipEntry.isDirectory()) {
if (!newFile.isDirectory() && !newFile.mkdirs()) {
throw new IOException("디렉토리 생성 실패: " + newFile);
}
} else {
// 상위 디렉토리가 없는 경우 생성
File parent = newFile.getParentFile();
if (!parent.exists() && !parent.mkdirs()) {
throw new IOException("상위 디렉토리 생성 실패: " + parent);
}
// 실제 파일 쓰기
try (FileOutputStream fos = new FileOutputStream(newFile)) {
byte[] buffer = new byte[1024];
int len;
while ((len = zis.read(buffer)) > 0) {
fos.write(buffer, 0, len);
}
}
}
zipEntry = zis.getNextEntry();
}
zis.closeEntry();
} catch (IOException e) {
throw new CustomApiException(
ApiResponseCode.INTERNAL_SERVER_ERROR.getId(),
HttpStatus.INTERNAL_SERVER_ERROR,
"압축 해제 중 오류가 발생했습니다: " + e.getMessage());
}
}
public static File newFile(File destinationDir, ZipEntry zipEntry) throws IOException {
File destFile = new File(destinationDir, zipEntry.getName());
String destDirPath = destinationDir.getCanonicalPath();
String destFilePath = destFile.getCanonicalPath();
if (!destFilePath.startsWith(destDirPath + File.separator)) {
throw new IOException("엔트리가 대상 디렉토리를 벗어남: " + zipEntry.getName());
}
return destFile;
}
public static List<String> execCommandAndReadLines(String command) {
List<String> result = new ArrayList<>();
String host = "192.168.2.86";
String user = "kcomu";
String password = "Kamco2025!";
Session session = null;
ChannelExec channel = null;
try {
JSch jsch = new JSch();
session = jsch.getSession(user, host, 22);
session.setPassword(password);
Properties config = new Properties();
config.put("StrictHostKeyChecking", "no");
session.setConfig(config);
session.connect(10_000);
channel = (ChannelExec) session.openChannel("exec");
channel.setCommand(command);
channel.setInputStream(null);
InputStream in = channel.getInputStream();
channel.connect();
try (BufferedReader br = new BufferedReader(new InputStreamReader(in))) {
String line;
while ((line = br.readLine()) != null) {
result.add(line);
}
}
return result;
} catch (Exception e) {
throw new RuntimeException("remote command failed : " + command, e);
} finally {
if (channel != null) channel.disconnect();
if (session != null) session.disconnect();
}
}
/** ✅ 폴더 재귀 삭제 */
private static void deleteDirectoryRecursively(Path path) throws IOException {
if (!Files.exists(path)) return;
// 하위부터 지워야 하므로 reverse order
Files.walk(path)
.sorted(Comparator.reverseOrder())
.forEach(
p -> {
try {
Files.deleteIfExists(p);
} catch (IOException e) {
// 여기서 바로 RuntimeException으로 올려서 상위 catch(IOException)로 잡히게 함
throw new UncheckedIOException("폴더 삭제 실패: " + p.toAbsolutePath(), e);
}
});
}
}

View File

@@ -0,0 +1,23 @@
package com.kamco.cd.training.common.utils;
import jakarta.servlet.http.HttpServletRequest;
public final class HeaderUtil {
private HeaderUtil() {}
/** 특정 Header 값 조회 */
public static String get(HttpServletRequest request, String headerName) {
if (request == null || headerName == null) {
return null;
}
String value = request.getHeader(headerName);
return (value != null && !value.isBlank()) ? value : null;
}
/** 필수 Header 조회 (없으면 null) */
public static String getRequired(HttpServletRequest request, String headerName) {
return get(request, headerName);
}
}

View File

@@ -0,0 +1,23 @@
package com.kamco.cd.training.config;
import java.util.concurrent.Executor;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
@Configuration
@EnableAsync
public class AsyncConfig {
@Bean(name = "trainJobExecutor")
public Executor trainJobExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(4); // 동시에 4개 실행
executor.setMaxPoolSize(8); // 최대 8개
executor.setQueueCapacity(200); // 대기 큐
executor.setThreadNamePrefix("train-job-");
executor.initialize();
return executor;
}
}

View File

@@ -76,11 +76,13 @@ public class SecurityConfig {
"/api/auth/logout",
"/swagger-ui/**",
"/v3/api-docs/**",
"/api/members/*/password",
"/api/upload/chunk-upload-dataset",
"/api/upload/chunk-upload-complete")
"/api/upload/chunk-upload-complete",
"/download_progress_test.html",
"/api/models/download/**")
.permitAll()
.requestMatchers("/api/members/*/password")
.authenticated()
// default
.anyRequest()
.authenticated())

View File

@@ -5,11 +5,14 @@ import com.kamco.cd.training.log.dto.EventType;
import com.kamco.cd.training.menu.dto.MenuDto;
import jakarta.servlet.http.HttpServletRequest;
import java.io.UnsupportedEncodingException;
import java.util.Comparator;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import lombok.extern.slf4j.Slf4j;
import org.springframework.web.util.ContentCachingRequestWrapper;
@Slf4j
public class ApiLogFunction {
// 클라이언트 IP 추출
@@ -34,6 +37,14 @@ public class ApiLogFunction {
return ip;
}
public static String getXFowardedForIp(HttpServletRequest request) {
String ip = request.getHeader("X-Forwarded-For");
if (ip != null) {
ip = ip.split(",")[0].trim();
}
return ip;
}
// 사용자 ID 추출 예시 (Spring Security 기준)
public static String getUserId(HttpServletRequest request) {
try {
@@ -47,20 +58,20 @@ public class ApiLogFunction {
String method = request.getMethod().toUpperCase();
String uri = request.getRequestURI().toLowerCase();
// URL 기반 DOWNLOAD/PRINT 분류
// URL 기반 DOWNLOAD/PRINT 분류 -> /download는 FileDownloadInterceptor로 옮김
if (uri.contains("/download") || uri.contains("/export")) {
return EventType.DOWNLOAD;
}
if (uri.contains("/print")) {
return EventType.PRINT;
return EventType.OTHER;
}
// 일반 CRUD
return switch (method) {
case "POST" -> EventType.CREATE;
case "GET" -> EventType.READ;
case "DELETE" -> EventType.DELETE;
case "PUT", "PATCH" -> EventType.UPDATE;
case "POST" -> EventType.ADDED;
case "GET" -> EventType.LIST;
case "DELETE" -> EventType.REMOVE;
case "PUT", "PATCH" -> EventType.MODIFIED;
default -> EventType.OTHER;
};
}
@@ -121,12 +132,22 @@ public class ApiLogFunction {
public static String getUriMenuInfo(List<MenuDto.Basic> menuList, String uri) {
MenuDto.Basic m =
String normalizedUri = uri.replace("/api", "");
MenuDto.Basic basic =
menuList.stream()
.filter(menu -> menu.getMenuApiUrl() != null && uri.contains(menu.getMenuApiUrl()))
.findFirst()
.filter(
menu -> menu.getMenuUrl() != null && normalizedUri.startsWith(menu.getMenuUrl()))
.max(Comparator.comparingInt(m -> m.getMenuUrl().length()))
.orElse(null);
return m != null ? m.getMenuUid() : "SYSTEM";
return basic != null ? basic.getMenuUid() : "SYSTEM";
}
public static String cutRequestBody(String value) {
int MAX_LEN = 255;
if (value == null) {
return null;
}
return value.length() <= MAX_LEN ? value : value.substring(0, MAX_LEN);
}
}

View File

@@ -2,10 +2,17 @@ package com.kamco.cd.training.config.api;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.kamco.cd.training.auth.CustomUserDetails;
import com.kamco.cd.training.common.utils.HeaderUtil;
import com.kamco.cd.training.log.dto.EventType;
import com.kamco.cd.training.menu.dto.MenuDto;
import com.kamco.cd.training.menu.service.MenuService;
import com.kamco.cd.training.postgres.entity.AuditLogEntity;
import com.kamco.cd.training.postgres.repository.log.AuditLogRepository;
import jakarta.servlet.http.HttpServletRequest;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Optional;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.MethodParameter;
import org.springframework.http.MediaType;
@@ -23,6 +30,7 @@ import org.springframework.web.util.ContentCachingRequestWrapper;
*
* <p>createOK() → 201 CREATED ok() → 200 OK deleteOk() → 204 NO_CONTENT
*/
@Slf4j
@RestControllerAdvice
public class ApiResponseAdvice implements ResponseBodyAdvice<Object> {
@@ -61,12 +69,27 @@ public class ApiResponseAdvice implements ResponseBodyAdvice<Object> {
if (body instanceof ApiResponseDto<?> apiResponse) {
response.setStatusCode(apiResponse.getHttpStatus());
String ip = ApiLogFunction.getClientIp(servletRequest);
Long userid = null;
String actionType = HeaderUtil.get(servletRequest, "kamco-action-type");
// actionType 이 없으면 로그 저장하지 않기 || download 는 FileDownloadInterceptor 에서 하기
// (file down URL prefix 추가는 WebConfig.java 에 하기)
if (actionType == null || actionType.equalsIgnoreCase("download")) {
return body;
}
if (servletRequest.getUserPrincipal() instanceof UsernamePasswordAuthenticationToken auth
&& auth.getPrincipal() instanceof CustomUserDetails customUserDetails) {
userid = customUserDetails.getMember().getId();
String ip =
Optional.ofNullable(HeaderUtil.get(servletRequest, "kamco-user-ip"))
.orElseGet(() -> ApiLogFunction.getXFowardedForIp(servletRequest));
Long userid = null;
String loginAttemptId = null;
// 로그인 시도할 때
if (servletRequest.getRequestURI().contains("/api/auth/signin")) {
loginAttemptId = HeaderUtil.get(servletRequest, "kamco-login-attempt-id");
} else {
if (servletRequest.getUserPrincipal() instanceof UsernamePasswordAuthenticationToken auth
&& auth.getPrincipal() instanceof CustomUserDetails customUserDetails) {
userid = customUserDetails.getMember().getId();
}
}
String requestBody;
@@ -84,17 +107,33 @@ public class ApiResponseAdvice implements ResponseBodyAdvice<Object> {
requestBody = maskSensitiveFields(requestBody);
}
List<?> list = menuService.getFindAll();
List<MenuDto.Basic> result =
list.stream()
.map(
item -> {
if (item instanceof LinkedHashMap<?, ?> map) {
return objectMapper.convertValue(map, MenuDto.Basic.class);
} else if (item instanceof MenuDto.Basic dto) {
return dto;
} else {
throw new IllegalStateException("Unsupported cache type: " + item.getClass());
}
})
.toList();
AuditLogEntity log =
new AuditLogEntity(
userid,
ApiLogFunction.getEventType(servletRequest),
EventType.fromName(actionType),
ApiLogFunction.isSuccessFail(apiResponse),
ApiLogFunction.getUriMenuInfo(
menuService.getFindAll(), servletRequest.getRequestURI()),
ApiLogFunction.getUriMenuInfo(result, servletRequest.getRequestURI()),
ip,
servletRequest.getRequestURI(),
requestBody,
apiResponse.getErrorLogUid());
ApiLogFunction.cutRequestBody(requestBody),
apiResponse.getErrorLogUid(),
null,
loginAttemptId);
auditLogRepository.save(log);
}

View File

@@ -164,6 +164,7 @@ public class ApiResponseDto<T> {
NOT_FOUND_USER_FOR_EMAIL("이메일로 유저를 찾을 수 없습니다."),
NOT_FOUND_USER("사용자를 찾을 수 없습니다."),
UNPROCESSABLE_ENTITY("이 데이터는 삭제할 수 없습니다."),
UNPROCESSABLE_ENTITY_UPDATE("이 데이터는 수정할 수 없습니다."),
LOGIN_ID_NOT_FOUND("아이디를 잘못 입력하셨습니다."),
LOGIN_PASSWORD_MISMATCH("비밀번호를 잘못 입력하셨습니다."),
LOGIN_PASSWORD_EXCEEDED("비밀번호 오류 횟수를 초과하여 이용하실 수 없습니다.\n로그인 오류에 대해 관리자에게 문의하시기 바랍니다."),

View File

@@ -1,4 +1,4 @@
package com.kamco.cd.training.config;
package com.kamco.cd.training.config.swagger;
import io.swagger.v3.oas.annotations.enums.SecuritySchemeType;
import io.swagger.v3.oas.annotations.security.SecurityScheme;

View File

@@ -0,0 +1,97 @@
package com.kamco.cd.training.config.swagger;
import jakarta.servlet.http.HttpServletRequest;
import java.nio.charset.StandardCharsets;
import org.springdoc.core.properties.SwaggerUiConfigProperties;
import org.springdoc.core.properties.SwaggerUiOAuthProperties;
import org.springdoc.core.providers.ObjectMapperProvider;
import org.springdoc.webmvc.ui.SwaggerIndexPageTransformer;
import org.springdoc.webmvc.ui.SwaggerIndexTransformer;
import org.springdoc.webmvc.ui.SwaggerWelcomeCommon;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.context.annotation.Profile;
import org.springframework.core.io.Resource;
import org.springframework.web.servlet.resource.ResourceTransformerChain;
import org.springframework.web.servlet.resource.TransformedResource;
@Profile({"local", "dev"})
@Configuration
public class SwaggerUiAutoAuthConfig {
@Bean
@Primary
public SwaggerIndexTransformer swaggerIndexTransformer(
SwaggerUiConfigProperties swaggerUiConfigProperties,
SwaggerUiOAuthProperties swaggerUiOAuthProperties,
SwaggerWelcomeCommon swaggerWelcomeCommon,
ObjectMapperProvider objectMapperProvider) {
SwaggerIndexPageTransformer delegate =
new SwaggerIndexPageTransformer(
swaggerUiConfigProperties,
swaggerUiOAuthProperties,
swaggerWelcomeCommon,
objectMapperProvider);
return new SwaggerIndexTransformer() {
private static final String TOKEN_KEY = "SWAGGER_ACCESS_TOKEN";
@Override
public Resource transform(
HttpServletRequest request, Resource resource, ResourceTransformerChain chain) {
try {
// 1) springdoc 기본 변환 먼저 적용
Resource transformed = delegate.transform(request, resource, chain);
String html =
new String(transformed.getInputStream().readAllBytes(), StandardCharsets.UTF_8);
String loginPathContains = "/api/auth/signin";
String inject =
"""
tagsSorter: (a, b) => {
const TOP = '인증(Auth)';
if (a === TOP && b !== TOP) return -1;
if (b === TOP && a !== TOP) return 1;
return a.localeCompare(b);
},
requestInterceptor: (req) => {
const token = localStorage.getItem('%s');
if (token) {
req.headers = req.headers || {};
req.headers['Authorization'] = 'Bearer ' + token;
}
return req;
},
responseInterceptor: async (res) => {
try {
const isLogin = (res?.url?.includes('%s') && res?.status === 200);
if (isLogin) {
const text = (typeof res.data === 'string') ? res.data : JSON.stringify(res.data);
const json = JSON.parse(text);
const token = json?.data?.accessToken;
if (token) {
localStorage.setItem('%s', token);
}
}
} catch (e) {}
return res;
},
"""
.formatted(TOKEN_KEY, loginPathContains, TOKEN_KEY);
html = html.replace("SwaggerUIBundle({", "SwaggerUIBundle({\n" + inject);
return new TransformedResource(transformed, html.getBytes(StandardCharsets.UTF_8));
} catch (Exception e) {
// 실패 시 원본 반환(문서 깨짐 방지)
return resource;
}
}
};
}
}

View File

@@ -2,6 +2,9 @@ package com.kamco.cd.training.dataset;
import com.kamco.cd.training.config.api.ApiResponseDto;
import com.kamco.cd.training.dataset.dto.DatasetDto;
import com.kamco.cd.training.dataset.dto.DatasetObjDto;
import com.kamco.cd.training.dataset.dto.DatasetObjDto.DatasetClass;
import com.kamco.cd.training.dataset.dto.DatasetObjDto.DatasetStorage;
import com.kamco.cd.training.dataset.service.DatasetService;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.Parameter;
@@ -11,9 +14,16 @@ import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.responses.ApiResponses;
import io.swagger.v3.oas.annotations.tags.Tag;
import jakarta.validation.Valid;
import java.io.IOException;
import java.nio.file.FileStore;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;
import java.util.UUID;
import lombok.RequiredArgsConstructor;
import org.springframework.core.io.Resource;
import org.springframework.data.domain.Page;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;
@Tag(name = "학습데이터 관리", description = "어드민 홈 > 학습데이터관리 > 전체데이터 API")
@@ -39,9 +49,12 @@ public class DatasetApiController {
})
@GetMapping
public ApiResponseDto<Page<DatasetDto.Basic>> searchDatasets(
@Parameter(description = "구분", example = "DELIVER(납품), PRODUCTION(제작)")
@Parameter(
description = "구분",
example = "",
schema = @Schema(allowableValues = {"DELIVER", "PRODUCTION"}))
@RequestParam(required = false)
String groupTitle,
String dataType,
@Parameter(description = "제목", example = "") @RequestParam(required = false) String title,
@Parameter(description = "페이지 번호 (0부터 시작)", example = "0") @RequestParam(defaultValue = "0")
int page,
@@ -49,7 +62,7 @@ public class DatasetApiController {
int size) {
DatasetDto.SearchReq searchReq = new DatasetDto.SearchReq();
searchReq.setTitle(title);
searchReq.setGroupTitle(groupTitle);
searchReq.setDataType(dataType);
searchReq.setPage(page);
searchReq.setSize(size);
return ApiResponseDto.ok(datasetService.searchDatasets(searchReq));
@@ -73,26 +86,6 @@ public class DatasetApiController {
return ApiResponseDto.ok(datasetService.getDatasetDetail(uuid));
}
@Operation(summary = "학습데이터 등록", description = "학습데이터 파일 업로드")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "201",
description = "등록 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = Long.class))),
@ApiResponse(responseCode = "400", description = "잘못된 요청 데이터", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@PostMapping("/register")
public ApiResponseDto<Long> registerDataset(
@RequestBody @Valid DatasetDto.RegisterReq registerReq) {
Long id = datasetService.registerDataset(registerReq);
return ApiResponseDto.createOK(id);
}
@Operation(summary = "학습데이터 수정", description = "학습데이터 제목, 메모 수정")
@ApiResponses(
value = {
@@ -129,8 +122,7 @@ public class DatasetApiController {
return ApiResponseDto.ok(uuid);
}
/*
@Operation(summary = "데이터셋 통계 요약", description = "선택 데이터셋의 통계를 요약합니다.")
@Operation(summary = "학습데이터 관리 목록 조회", description = "학습데이터 목록을 조회합니다.")
@ApiResponses(
value = {
@ApiResponse(
@@ -139,16 +131,121 @@ public class DatasetApiController {
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = DatasetDto.Summary.class))),
@ApiResponse(responseCode = "400", description = "잘못된 요청", content = @Content),
schema = @Schema(implementation = Page.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@PostMapping("/summary")
public ApiResponseDto<DatasetDto.Summary> getDatasetSummary(
@RequestBody @Valid DatasetDto.SummaryReq summaryReq) {
return ApiResponseDto.ok(datasetService.getDatasetSummary(summaryReq));
@GetMapping("/obj-list")
public ApiResponseDto<Page<DatasetObjDto.Basic>> searchDatasetObjectList(
@Parameter(description = "회차 uuid", example = "e9a6774b-4f81-4402-b080-51d27fac1f01")
@RequestParam(required = true)
UUID uuid,
@Parameter(description = "비교년도분류", example = "container") @RequestParam(required = false)
String compareClassCd,
@Parameter(description = "기준년도분류", example = "waste") @RequestParam(required = false)
String targetClassCd,
@Parameter(description = "도엽번호", example = "36713060") @RequestParam(required = false)
String mapSheetNum,
@RequestParam(defaultValue = "0") int page,
@RequestParam(defaultValue = "20") int size) {
DatasetObjDto.SearchReq searchReq = new DatasetObjDto.SearchReq();
searchReq.setUuid(uuid);
searchReq.setCompareClassCd(compareClassCd);
searchReq.setTargetClassCd(targetClassCd);
searchReq.setMapSheetNum(mapSheetNum);
searchReq.setPage(page);
searchReq.setSize(size);
return ApiResponseDto.ok(datasetService.searchDatasetObjectList(searchReq));
}
*/
@Operation(summary = "학습데이터 관리 obj 삭제", description = "학습데이터 관리 obj 삭제 API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "삭제 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = Page.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@DeleteMapping("/obj/{uuid}")
public ApiResponseDto<UUID> deleteDatasetObjByUuid(@PathVariable UUID uuid) {
return ApiResponseDto.ok(datasetService.deleteDatasetObjByUuid(uuid));
}
@Operation(summary = "학습데이터 결과 class 조회", description = "학습데이터 결과 class 조회 API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = Page.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/class/{uuid}")
public ApiResponseDto<List<DatasetClass>> getDatasetObjByUuid(
@Parameter(description = "dataset uuid", example = "e1416f32-769f-495c-a883-3ebfacef4bac")
@PathVariable
UUID uuid,
@Parameter(description = "compare, target", example = "compare") @RequestParam String type) {
return ApiResponseDto.ok(datasetService.getDatasetObjByUuid(uuid, type));
}
@Operation(summary = "남은 저장공간 조회", description = "남은 저장공간 조회 API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = Page.class))),
@ApiResponse(responseCode = "404", description = "저장 공간 조회 오류", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/usable-bytes")
public ApiResponseDto<DatasetStorage> getUsableBytes() throws IOException {
FileStore store = Files.getFileStore(Path.of("."));
long usable = store.getUsableSpace();
DatasetStorage storage = new DatasetStorage();
storage.setUsableBytes(String.valueOf(usable));
// datasetService.getUsableBytes();
return ApiResponseDto.ok(storage);
}
@Operation(summary = "학습데이터 zip파일 등록", description = "학습데이터 zip파일 등록 합니다.")
@PostMapping
public ApiResponseDto<ApiResponseDto.ResponseObj> insertDataset(
@RequestBody @Valid DatasetDto.AddReq addReq) {
return ApiResponseDto.okObject(datasetService.insertDataset(addReq));
}
@Operation(summary = "객체별 파일 Path 조회", description = "파일 Path 조회")
@GetMapping("/files")
public ResponseEntity<Resource> getFile(@RequestParam UUID uuid, @RequestParam String pathType)
throws Exception {
String path = datasetService.getFilePathByUUIDPathType(uuid, pathType);
return datasetService.getFilePathByFile(path);
}
@Operation(summary = "객체별 파일 Path 조회", description = "파일 Path 조회")
@GetMapping("/files-to86")
public ResponseEntity<Resource> getFileTo86(
@RequestParam UUID uuid, @RequestParam String pathType) throws Exception {
String path = datasetService.getFilePathByUUIDPathType(uuid, pathType);
return datasetService.getFilePathByFile(path);
}
}

View File

@@ -1,56 +0,0 @@
package com.kamco.cd.training.dataset;
import com.kamco.cd.training.config.api.ApiResponseDto;
import com.kamco.cd.training.dataset.dto.MapSheetDto;
import com.kamco.cd.training.dataset.service.MapSheetService;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.responses.ApiResponses;
import io.swagger.v3.oas.annotations.tags.Tag;
import jakarta.validation.Valid;
import lombok.RequiredArgsConstructor;
import org.springframework.data.domain.Page;
import org.springframework.web.bind.annotation.*;
@Tag(name = "도엽 관리", description = "도엽(MapSheet) 관리 API")
@RestController
@RequiredArgsConstructor
public class MapSheetApiController {
private final MapSheetService mapSheetService;
@Operation(summary = "도엽 목록 조회", description = "데이터셋의 도엽 목록을 조회합니다.")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = Page.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@PostMapping("/api/datasets/items/search")
public ApiResponseDto<Page<MapSheetDto.Basic>> searchMapSheets(
@RequestBody @Valid MapSheetDto.SearchReq searchReq) {
return ApiResponseDto.ok(mapSheetService.searchMapSheets(searchReq));
}
@Operation(summary = "도엽 삭제", description = "도엽을 삭제합니다 (다건 지원).")
@ApiResponses(
value = {
@ApiResponse(responseCode = "200", description = "삭제 성공", content = @Content),
@ApiResponse(responseCode = "400", description = "잘못된 요청", content = @Content),
@ApiResponse(responseCode = "404", description = "도엽을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@PostMapping("/api/datasets/items/delete")
public ApiResponseDto<Void> deleteMapSheets(@RequestBody @Valid MapSheetDto.DeleteReq deleteReq) {
mapSheetService.deleteMapSheets(deleteReq);
return ApiResponseDto.ok(null);
}
}

View File

@@ -1,7 +1,9 @@
package com.kamco.cd.training.dataset.dto;
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.kamco.cd.training.common.enums.LearnDataRegister;
import com.kamco.cd.training.common.enums.LearnDataType;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.common.utils.enums.Enums;
import com.kamco.cd.training.common.utils.interfaces.JsonFormatDttm;
import io.swagger.v3.oas.annotations.media.Schema;
@@ -12,13 +14,16 @@ import java.time.ZonedDateTime;
import java.util.List;
import java.util.UUID;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import lombok.extern.slf4j.Slf4j;
import org.springframework.data.domain.PageRequest;
import org.springframework.data.domain.Pageable;
import org.springframework.data.domain.Sort;
@Slf4j
public class DatasetDto {
@Schema(name = "Dataset Basic", description = "데이터셋 기본 정보")
@@ -30,57 +35,72 @@ public class DatasetDto {
private Long id;
private UUID uuid;
private String groupTitle;
private String groupTitleCd;
private String title;
private Long roundNo;
private Integer compareYyyy;
private Integer targetYyyy;
private String totalSize;
private String memo;
@JsonFormatDttm private ZonedDateTime createdDttm;
private String status;
private String statusCd;
private Boolean deleted;
private String dataType;
public Basic(
Long id,
UUID uuid,
String groupTitle,
String title,
Long roundNo,
Integer compareYyyy,
Integer targetYyyy,
Long totalSize,
String memo,
ZonedDateTime createdDttm,
String status,
Boolean deleted) {
Boolean deleted,
String dataType) {
this.id = id;
this.uuid = uuid;
this.groupTitle = getGroupTitle(groupTitle);
this.groupTitleCd = groupTitle;
this.title = title;
this.roundNo = roundNo;
this.compareYyyy = compareYyyy;
this.targetYyyy = targetYyyy;
this.totalSize = getTotalSize(totalSize);
this.memo = memo;
this.createdDttm = createdDttm;
this.status = getStatus(status);
this.statusCd = status;
this.deleted = deleted;
this.dataType = dataType;
}
public String getTotalSize(Long totalSize) {
if (totalSize == null) return "0G";
double giga = totalSize / (1024.0 * 1024 * 1024);
return String.format("%.2fG", giga);
}
if (totalSize == null || totalSize <= 0) return "0M";
public String getGroupTitle(String groupTitleCd) {
LearnDataType type = Enums.fromId(LearnDataType.class, groupTitleCd);
return type == null ? null : type.getText();
double giga = totalSize / (1024.0 * 1024 * 1024);
if (giga >= 1) {
return String.format("%.2fG", giga);
} else {
double mega = totalSize / (1024.0 * 1024);
return String.format("%.2fM", mega);
}
}
public String getStatus(String status) {
LearnDataRegister type = Enums.fromId(LearnDataRegister.class, status);
return type == null ? null : type.getText();
}
public String getYear() {
return this.compareYyyy + "-" + this.targetYyyy;
}
public String getDataTypeName() {
LearnDataType type = Enums.fromId(LearnDataType.class, this.dataType);
return type == null ? null : type.getText();
}
}
@Schema(name = "Dataset Detail", description = "데이터셋 상세 정보")
@@ -108,8 +128,8 @@ public class DatasetDto {
@AllArgsConstructor
public static class SearchReq {
@Schema(description = "구분", example = "DELIVER(납품), PRODUCTION(제작)")
private String groupTitle;
@Schema(description = "구분")
private String dataType;
@Schema(description = "제목 (부분 검색)", example = "1차")
private String title;
@@ -121,9 +141,7 @@ public class DatasetDto {
private int size = 20;
public Pageable toPageable() {
// API에서는 1부터 시작하지만 내부적으로는 0부터 시작
int pageIndex = Math.max(0, page - 1);
return PageRequest.of(pageIndex, size, Sort.by(Sort.Direction.DESC, "createdDttm"));
return PageRequest.of(page, size, Sort.by(Sort.Direction.DESC, "createdDttm"));
}
}
@@ -151,10 +169,11 @@ public class DatasetDto {
@Schema(description = "제목", example = "1차 제작")
private String title;
@NotBlank(message = "연도는 필수입니다")
@Size(max = 4, message = "연도는 4자리입니다")
@Schema(description = "연도 (YYYY)", example = "2024")
private String year;
@Schema(description = "비교연도 (YYYY)", example = "2023")
private Integer compareYear;
@Schema(description = "기준연도 (YYYY)", example = "2024")
private Integer targetYyyy;
@Schema(description = "회차", example = "1")
private Long roundNo;
@@ -209,4 +228,308 @@ public class DatasetDto {
@Schema(description = "평균 도엽 수", example = "750")
private double averageMapSheets;
}
@Schema(name = "SelectDataSet", description = "데이터셋 선택 리스트")
@Getter
@Setter
@NoArgsConstructor
public static class SelectDataSet {
private String modelNo; // G1, G2, G3 모델 타입
private Long datasetId;
private UUID uuid;
private String dataType;
private String title;
private Long roundNo;
private Integer compareYyyy;
private Integer targetYyyy;
private String memo;
@JsonIgnore private Long classCount;
private Integer buildingCnt;
private Integer containerCnt;
private String dataTypeName;
private Long wasteCnt;
private Long landCoverCnt;
public SelectDataSet(
String modelNo,
Long datasetId,
UUID uuid,
String dataType,
String title,
Long roundNo,
Integer compareYyyy,
Integer targetYyyy,
String memo,
Long classCount) {
this.datasetId = datasetId;
this.uuid = uuid;
this.dataType = dataType;
this.dataTypeName = getDataTypeName(dataType);
this.title = title;
this.roundNo = roundNo;
this.compareYyyy = compareYyyy;
this.targetYyyy = targetYyyy;
this.memo = memo;
this.classCount = classCount;
if (modelNo.equals(ModelType.G2.getId())) {
this.wasteCnt = classCount;
} else if (modelNo.equals(ModelType.G3.getId())) {
this.landCoverCnt = classCount;
}
}
public SelectDataSet(
String modelNo,
Long datasetId,
UUID uuid,
String dataType,
String title,
Long roundNo,
Integer compareYyyy,
Integer targetYyyy,
String memo,
Integer buildingCnt,
Integer containerCnt) {
this.datasetId = datasetId;
this.uuid = uuid;
this.dataType = dataType;
this.dataTypeName = getDataTypeName(dataType);
this.title = title;
this.roundNo = roundNo;
this.compareYyyy = compareYyyy;
this.targetYyyy = targetYyyy;
this.memo = memo;
this.buildingCnt = buildingCnt;
this.containerCnt = containerCnt;
}
public String getDataTypeName(String groupTitleCd) {
LearnDataType type = Enums.fromId(LearnDataType.class, groupTitleCd);
return type == null ? null : type.getText();
}
public String getYear() {
return this.compareYyyy + "-" + this.targetYyyy;
}
}
@Schema(name = "SelectTransferDataSet", description = "전이학습 데이터셋 선택 리스트")
@Getter
@Setter
@NoArgsConstructor
public static class SelectTransferDataSet {
private String modelNo; // G1, G2, G3 모델 타입
private Long datasetId;
private UUID uuid;
private String dataType;
private String title;
private Long roundNo;
private Integer compareYyyy;
private Integer targetYyyy;
private String memo;
@JsonIgnore private Long classCount;
private Integer buildingCnt;
private Integer containerCnt;
private String dataTypeName;
private Long wasteCnt;
private Long landCoverCnt;
private String beforeModelNo; // G1, G2, G3 모델 타입
private Long beforeDatasetId;
private UUID beforeUuid;
private String beforeDataType;
private String beforeTitle;
private Long beforeRoundNo;
private Integer beforeCompareYyyy;
private Integer beforeTargetYyyy;
private String beforeMemo;
@JsonIgnore private Long beforeClassCount;
private Integer beforeBuildingCnt;
private Integer beforeContainerCnt;
private String beforeDataTypeName;
private Long beforeWasteCnt;
private Long beforeLandCoverCnt;
public SelectTransferDataSet(
// 현재
String modelNo,
Long datasetId,
UUID uuid,
String dataType,
String title,
Long roundNo,
Integer compareYyyy,
Integer targetYyyy,
String memo,
Long classCount,
// 이전(before)
String beforeModelNo,
Long beforeDatasetId,
UUID beforeUuid,
String beforeDataType,
String beforeTitle,
Long beforeRoundNo,
Integer beforeCompareYyyy,
Integer beforeTargetYyyy,
String beforeMemo,
Long beforeClassCount) {
// 현재
this.modelNo = modelNo;
this.datasetId = datasetId;
this.uuid = uuid;
this.dataType = dataType;
this.dataTypeName = getDataTypeName(dataType);
this.title = title;
this.roundNo = roundNo;
this.compareYyyy = compareYyyy;
this.targetYyyy = targetYyyy;
this.memo = memo;
this.classCount = classCount;
if (modelNo != null && modelNo.equals(ModelType.G2.getId())) {
this.wasteCnt = classCount;
} else if (modelNo != null && modelNo.equals(ModelType.G3.getId())) {
this.landCoverCnt = classCount;
}
// 이전(before)
this.beforeModelNo = beforeModelNo;
this.beforeDatasetId = beforeDatasetId;
this.beforeUuid = beforeUuid;
this.beforeDataType = beforeDataType;
this.beforeDataTypeName = getDataTypeName(beforeDataType);
this.beforeTitle = beforeTitle;
this.beforeRoundNo = beforeRoundNo;
this.beforeCompareYyyy = beforeCompareYyyy;
this.beforeTargetYyyy = beforeTargetYyyy;
this.beforeMemo = beforeMemo;
this.beforeClassCount = beforeClassCount;
if (beforeModelNo != null && beforeModelNo.equals(ModelType.G2.getId())) {
this.beforeWasteCnt = beforeClassCount;
} else if (beforeModelNo != null && beforeModelNo.equals(ModelType.G3.getId())) {
this.beforeLandCoverCnt = beforeClassCount;
}
}
public SelectTransferDataSet(
// 현재
String modelNo,
Long datasetId,
UUID uuid,
String dataType,
String title,
Long roundNo,
Integer compareYyyy,
Integer targetYyyy,
String memo,
Integer buildingCnt,
Integer containerCnt,
// 이전(before)
String beforeModelNo,
Long beforeDatasetId,
UUID beforeUuid,
String beforeDataType,
String beforeTitle,
Long beforeRoundNo,
Integer beforeCompareYyyy,
Integer beforeTargetYyyy,
String beforeMemo,
Integer beforeBuildingCnt,
Integer beforeContainerCnt) {
// 현재
this.modelNo = modelNo;
this.datasetId = datasetId;
this.uuid = uuid;
this.dataType = dataType;
this.dataTypeName = getDataTypeName(dataType);
this.title = title;
this.roundNo = roundNo;
this.compareYyyy = compareYyyy;
this.targetYyyy = targetYyyy;
this.memo = memo;
this.buildingCnt = buildingCnt;
this.containerCnt = containerCnt;
// 이전(before)
this.beforeModelNo = beforeModelNo;
this.beforeDatasetId = beforeDatasetId;
this.beforeUuid = beforeUuid;
this.beforeDataType = beforeDataType;
this.beforeDataTypeName = getDataTypeName(beforeDataType);
this.beforeTitle = beforeTitle;
this.beforeRoundNo = beforeRoundNo;
this.beforeCompareYyyy = beforeCompareYyyy;
this.beforeTargetYyyy = beforeTargetYyyy;
this.beforeMemo = beforeMemo;
this.beforeBuildingCnt = beforeBuildingCnt;
this.beforeContainerCnt = beforeContainerCnt;
}
public String getDataTypeName(String groupTitleCd) {
LearnDataType type = Enums.fromId(LearnDataType.class, groupTitleCd);
return type == null ? null : type.getText();
}
public String getYear() {
return this.compareYyyy + "-" + this.targetYyyy;
}
public String getBeforeYear() {
if (this.beforeCompareYyyy == null || this.beforeTargetYyyy == null) {
return null;
}
return this.beforeCompareYyyy + "-" + this.beforeTargetYyyy;
}
}
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class DatasetReq {
String modelNo;
String dataType;
UUID uuid;
Long id;
List<Long> ids;
}
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class AddReq {
private String fileName;
private String filePath;
private Long fileSize;
private String memo;
}
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class DatasetMngRegDto {
private String uid;
private String dataType;
private Integer compareYyyy;
private Integer targetYyyy;
private Long roundNo;
private String title;
private String memo;
private Long totalSize;
private Long totalObjectCount;
private String datasetPath;
}
}

View File

@@ -0,0 +1,163 @@
package com.kamco.cd.training.dataset.dto;
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.kamco.cd.training.common.enums.DetectionClassification;
import com.kamco.cd.training.common.utils.interfaces.JsonFormatDttm;
import io.swagger.v3.oas.annotations.media.Schema;
import java.time.ZonedDateTime;
import java.util.UUID;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import lombok.extern.slf4j.Slf4j;
import org.springframework.data.domain.PageRequest;
import org.springframework.data.domain.Pageable;
import org.springframework.data.domain.Sort;
@Slf4j
public class DatasetObjDto {
@Schema(name = "DatasetObj Basic", description = "데이터셋 객체 Obj 기본 정보")
@Getter
@Setter
@NoArgsConstructor
public static class Basic {
private Long objId;
private Long datasetUid;
private Integer targetYyyy;
private String targetClassCd;
private Integer compareYyyy;
private String compareClassCd;
private String targetPath;
private String comparePath;
private String labelPath;
private String geojsonPath;
private String mapSheetNum;
@JsonFormatDttm private ZonedDateTime createdDttm;
private Long createdUid;
private Boolean deleted;
private UUID uuid;
@JsonIgnore private String geoJsonb;
private JsonNode geoJson;
public Basic(
Long objId,
Long datasetUid,
Integer targetYyyy,
String targetClassCd,
Integer compareYyyy,
String compareClassCd,
String targetPath,
String comparePath,
String labelPath,
String geojsonPath,
String mapSheetNum,
ZonedDateTime createdDttm,
Long createdUid,
Boolean deleted,
UUID uuid,
String geoJsonb) {
this.objId = objId;
this.datasetUid = datasetUid;
this.targetYyyy = targetYyyy;
this.targetClassCd = targetClassCd;
this.compareYyyy = compareYyyy;
this.compareClassCd = compareClassCd;
this.targetPath = targetPath;
this.comparePath = comparePath;
this.labelPath = labelPath;
this.geojsonPath = geojsonPath;
this.mapSheetNum = mapSheetNum;
this.createdDttm = createdDttm;
this.createdUid = createdUid;
this.deleted = deleted;
this.uuid = uuid;
this.geoJsonb = geoJsonb;
JsonNode geoJsonNode = null;
ObjectMapper mapper = new ObjectMapper();
if (geoJsonb != null) {
try {
geoJsonNode = mapper.readTree(geoJsonb);
} catch (JsonProcessingException e) {
throw new RuntimeException(e);
}
}
this.geoJson = geoJsonNode;
}
}
@Schema(name = "DatasetSearchReq", description = "데이터셋 상세 도엽목록 조회 요청")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class SearchReq {
@Schema(description = "회차 uuid", example = "e9a6774b-4f81-4402-b080-51d27fac1f01")
private UUID uuid;
@Schema(description = "비교년도분류", example = "waste")
private String compareClassCd;
@Schema(description = "기준년도분류", example = "land")
private String targetClassCd;
@Schema(description = "도엽번호", example = "36713060")
private String mapSheetNum;
@Schema(description = "페이지 번호 (0부터 시작)", example = "0")
private int page = 0;
@Schema(description = "페이지 크기", example = "20")
private int size = 20;
public Pageable toPageable() {
return PageRequest.of(page, size, Sort.by(Sort.Direction.DESC, "createdDttm"));
}
}
@Getter
@Setter
@AllArgsConstructor
@NoArgsConstructor
public static class DatasetClass {
private String classCd;
public String getClassName() {
return DetectionClassification.valueOf(classCd.toUpperCase()).getDesc();
}
}
@Getter
@Setter
public static class DatasetStorage {
private String usableBytes;
}
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class DatasetObjRegDto {
private Long datasetUid;
private Integer compareYyyy;
private String compareClassCd;
private Integer targetYyyy;
private String targetClassCd;
private String comparePath;
private String targetPath;
private String labelPath;
private String geojsonPath;
private String mapSheetNum;
private JsonNode geojson;
private String fileName;
}
}

View File

@@ -1,22 +1,68 @@
package com.kamco.cd.training.dataset.service;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.ObjectNode;
import com.kamco.cd.training.common.enums.LearnDataType;
import com.kamco.cd.training.common.exception.CustomApiException;
import com.kamco.cd.training.common.service.FormatStorage;
import com.kamco.cd.training.common.utils.FIleChecker;
import com.kamco.cd.training.config.api.ApiResponseDto.ApiResponseCode;
import com.kamco.cd.training.config.api.ApiResponseDto.ResponseObj;
import com.kamco.cd.training.dataset.dto.DatasetDto;
import com.kamco.cd.training.dataset.dto.DatasetDto.AddReq;
import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetMngRegDto;
import com.kamco.cd.training.dataset.dto.DatasetObjDto;
import com.kamco.cd.training.dataset.dto.DatasetObjDto.DatasetClass;
import com.kamco.cd.training.dataset.dto.DatasetObjDto.DatasetObjRegDto;
import com.kamco.cd.training.dataset.dto.DatasetObjDto.DatasetStorage;
import com.kamco.cd.training.dataset.dto.DatasetObjDto.SearchReq;
import com.kamco.cd.training.postgres.core.DatasetCoreService;
import jakarta.validation.Valid;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.UUID;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.InputStreamResource;
import org.springframework.core.io.Resource;
import org.springframework.data.domain.Page;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
@Slf4j
@Service
@RequiredArgsConstructor
@Transactional(readOnly = true)
@Transactional
public class DatasetService {
private final DatasetCoreService datasetCoreService;
@Value("${file.dataset-dir}")
private String datasetDir;
private static final List<String> LABEL_DIRS = List.of("label-json", "label", "input1", "input2");
private static final List<String> REQUIRED_DIRS = Arrays.asList("train", "val", "test");
private static final List<String> CHECK_DIRS = List.of("label", "input1", "input2");
/**
* 데이터셋 목록 조회
*
@@ -83,4 +129,399 @@ public class DatasetService {
log.info("데이터셋 통계 요약 - 요청: {}", summaryReq);
return datasetCoreService.getDatasetSummary(summaryReq);
}
public Page<DatasetObjDto.Basic> searchDatasetObjectList(SearchReq searchReq) {
return datasetCoreService.searchDatasetObjectList(searchReq);
}
public UUID deleteDatasetObjByUuid(UUID uuid) {
return datasetCoreService.deleteDatasetObjByUuid(uuid);
}
/**
* 데이터셋 object class 조회
*
* @param uuid dataset uuid
* @param type compare, target
*/
public List<DatasetClass> getDatasetObjByUuid(UUID uuid, String type) {
return datasetCoreService.findDatasetObjClassByUuid(uuid, type);
}
/**
* 사용 가능 공간 조회
*
* @return
*/
public DatasetStorage getUsableBytes() {
// 현재 실행 위치가 속한 디스크 기준
try {
FormatStorage.DiskUsage usage = FormatStorage.getDiskUsage(Path.of("."));
log.debug("경로 : {}", usage.path());
log.debug("총 저장공간 : {}", usage.totalText());
log.debug("남은 저장공간 : {}", usage.usableText());
log.debug("사용률 : {}", usage.usedPercent());
DatasetStorage datasetStorage = new DatasetStorage();
datasetStorage.setUsableBytes(usage.usableText());
return datasetStorage;
} catch (Exception e) {
throw new CustomApiException("NOT_FOUND", HttpStatus.NOT_FOUND);
}
}
@Transactional
public ResponseObj insertDataset(@Valid AddReq addReq) {
Long datasetUid = null; // master id 값, 등록하면서 가져올 예정
try {
// 같은 uid 로 등록한 파일이 있는지 확인
Long existsCnt =
datasetCoreService.findDatasetByUidExistsCnt(addReq.getFileName().replace(".zip", ""));
if (existsCnt > 0) {
return new ResponseObj(ApiResponseCode.DUPLICATE_DATA, "이미 등록된 회차 데이터 파일입니다. 확인 부탁드립니다.");
}
// 압축 해제
FIleChecker.unzip(addReq.getFileName(), addReq.getFilePath());
// 압축 해제한 폴더 하위에 train,val,test 폴더 모두 존재하는지 확인
validateTrainValTestDirs(addReq.getFilePath() + addReq.getFileName().replace(".zip", ""));
// 압축 해제한 폴더의 갯수 맞는지 log 찍기
validateDirFileCount(addReq.getFilePath() + addReq.getFileName().replace(".zip", ""));
// 해제한 폴더 읽어서 데이터 저장
List<Map<String, Object>> list =
getUnzipDatasetFiles(
addReq.getFilePath() + addReq.getFileName().replace(".zip", ""), "train");
int idx = 0;
for (Map<String, Object> map : list) {
datasetUid =
this.insertTrainTestData(map, addReq, idx, datasetUid, "train"); // train 데이터 insert
idx++;
}
List<Map<String, Object>> valList =
getUnzipDatasetFiles(
addReq.getFilePath() + addReq.getFileName().replace(".zip", ""), "val");
int valIdx = 0;
for (Map<String, Object> valid : valList) {
datasetUid =
this.insertTrainTestData(valid, addReq, valIdx, datasetUid, "val"); // val 데이터 insert
valIdx++;
}
List<Map<String, Object>> testList =
getUnzipDatasetFiles(
addReq.getFilePath() + addReq.getFileName().replace(".zip", ""), "test");
int testIdx = 0;
for (Map<String, Object> test : testList) {
datasetUid =
this.insertTrainTestData(test, addReq, testIdx, datasetUid, "test"); // test 데이터 insert
testIdx++;
}
} catch (IOException e) {
log.error(e.getMessage());
return new ResponseObj(ApiResponseCode.INTERNAL_SERVER_ERROR, e.getMessage());
}
datasetCoreService.updateDatasetUploadStatus(datasetUid);
return new ResponseObj(ApiResponseCode.OK, "업로드 성공하였습니다.");
}
@Transactional
public Long insertTrainTestData(
Map<String, Object> map, AddReq addReq, int idx, Long datasetUid, String subDir) {
ObjectMapper mapper = new ObjectMapper();
String comparePath = (String) map.get("input1");
String targetPath = (String) map.get("input2");
String labelPath = (String) map.get("label");
String geojsonPath = (String) map.get("geojson_path");
Object labelJson = map.get("label-json");
JsonNode json;
if (labelJson instanceof JsonNode jn) {
json = jn;
} else {
try {
json = mapper.readTree(labelJson.toString());
} catch (IOException e) {
throw new RuntimeException("label_json parse error", e);
}
}
String fileName = Paths.get(comparePath).getFileName().toString();
String[] fileNameStr = fileName.split("_");
String compareYyyy = fileNameStr[1];
String targetYyyy = fileNameStr[2];
String mapSheetNum = fileNameStr[3];
if (idx == 0 && subDir.equals("train")) {
String title = compareYyyy + "-" + targetYyyy; // 제목 : 비교년도-기준년도
String dataType = LearnDataType.PRODUCTION.getId(); // 만들어 넣는 건 다 제작
Long stage =
datasetCoreService.getDatasetMaxStage(
Integer.parseInt(compareYyyy), Integer.parseInt(targetYyyy))
+ 1;
String uid = addReq.getFileName().replace(".zip", "");
DatasetMngRegDto mngRegDto =
DatasetMngRegDto.builder()
.uid(uid)
.dataType(dataType)
.compareYyyy(Integer.parseInt(compareYyyy))
.targetYyyy(Integer.parseInt(targetYyyy))
.title(title)
.memo(addReq.getMemo())
.roundNo(stage)
.totalSize(addReq.getFileSize())
.datasetPath(addReq.getFilePath())
.build();
datasetUid = datasetCoreService.insertDatasetMngData(mngRegDto); // tb_dataset 에 insert
}
// datasetUid 로 obj 도 등록하기
// Json 갯수만큼 for문 돌려서 insert 해야 함, features에 빈값
if (json != null && json.path("features") != null && !json.path("features").isEmpty()) {
for (JsonNode feature : json.path("features")) {
JsonNode prop = feature.path("properties");
String compareClassCd = prop.path("before").asText(null);
String targetClassCd = prop.path("after").asText(null);
// 한 개씩 자른 geojson을 FeatureCollection 으로 만들어서 넣기
ObjectNode root = mapper.createObjectNode();
root.put("type", "FeatureCollection");
ArrayNode features = mapper.createArrayNode();
features.add(feature);
root.set("features", features);
DatasetObjRegDto objRegDto =
DatasetObjRegDto.builder()
.datasetUid(datasetUid)
.compareYyyy(Integer.parseInt(compareYyyy))
.compareClassCd(compareClassCd)
.targetYyyy(Integer.parseInt(targetYyyy))
.targetClassCd(targetClassCd)
.comparePath(comparePath)
.targetPath(targetPath)
.labelPath(labelPath)
.mapSheetNum(mapSheetNum)
.geojson(root)
.geojsonPath(geojsonPath)
.fileName(fileName)
.build();
if (subDir.equals("train")) {
datasetCoreService.insertDatasetObj(objRegDto);
} else if (subDir.equals("val")) {
datasetCoreService.insertDatasetValObj(objRegDto);
} else {
datasetCoreService.insertDatasetTestObj(objRegDto);
}
}
}
return datasetUid;
}
private List<Map<String, Object>> getUnzipDatasetFiles(String unzipRootPath, String subDir) {
Path root = Paths.get(unzipRootPath).resolve(subDir);
Map<String, Map<String, Object>> grouped = new HashMap<>();
for (String dirName : LABEL_DIRS) {
Path dir = root.resolve(dirName);
if (!Files.isDirectory(dir)) {
throw new CustomApiException(
ApiResponseCode.NOT_FOUND_DATA.getId(),
HttpStatus.CONFLICT,
"폴더가 존재하지 않습니다. 업로드 된 파일을 확인하세요. : " + dir);
}
try (Stream<Path> stream = Files.list(dir)) {
stream
.filter(Files::isRegularFile)
.forEach(
path -> {
String fileName = path.getFileName().toString();
String baseName = getBaseName(fileName);
// baseName 기준 Map 생성
Map<String, Object> data =
grouped.computeIfAbsent(baseName, k -> new HashMap<>());
// 공통 메타
data.put("baseName", baseName);
// 폴더별 처리
if ("label-json".equals(dirName)) {
// json 파일이면 파싱
data.put("label-json", readJson(path));
data.put("geojson_path", path.toAbsolutePath().toString());
} else {
data.put(dirName, path.toAbsolutePath().toString());
}
});
} catch (IOException e) {
throw new RuntimeException(e);
}
}
return new ArrayList<>(grouped.values());
}
private String getBaseName(String fileName) {
int idx = fileName.lastIndexOf('.');
return (idx > 0) ? fileName.substring(0, idx) : fileName;
}
private JsonNode readJson(Path jsonPath) {
try {
ObjectMapper mapper = new ObjectMapper();
return mapper.readTree(jsonPath.toFile());
} catch (IOException e) {
throw new RuntimeException("JSON 읽기 실패: " + jsonPath, e);
}
}
public String getFilePathByUUIDPathType(UUID uuid, String pathType) {
return datasetCoreService.getFilePathByUUIDPathType(uuid, pathType);
}
private String readRemoteFileAsString(String remoteFilePath) {
String command = "cat " + escape(remoteFilePath);
List<String> lines = FIleChecker.execCommandAndReadLines(command);
return String.join("\n", lines);
}
private JsonNode parseJson(String json) {
try {
ObjectMapper mapper = new ObjectMapper();
return mapper.readTree(json);
} catch (IOException e) {
throw new RuntimeException("JSON 파싱 실패", e);
}
}
private String escape(String path) {
// 쉘 커맨드에서 안전하게 사용할 수 있도록 문자열을 작은따옴표로 감싸면서, 내부의 작은따옴표를 이스케이프 처리
return "'" + path.replace("'", "'\"'\"'") + "'";
}
private static String normalizeLinuxPath(String path) {
return path.replace("\\", "/");
}
public ResponseEntity<Resource> getFilePathByFile(String remoteFilePath) {
try {
Path path = Paths.get(remoteFilePath);
InputStream inputStream = Files.newInputStream(path);
InputStreamResource resource =
new InputStreamResource(inputStream) {
@Override
public long contentLength() {
return -1; // 알 수 없으면 -1
}
};
String fileName = Paths.get(remoteFilePath.replace("\\", "/")).getFileName().toString();
return ResponseEntity.ok()
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + fileName + "\"")
.contentType(MediaType.APPLICATION_OCTET_STREAM)
.body(resource);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
/** unzipRootDir: 압축 해제된 폴더 경로 (ex: /data/xxx/myzipname) */
public static void validateTrainValTestDirs(String unzipRootDir) {
Path root = Paths.get(unzipRootDir);
// 루트 폴더 자체 존재 확인
if (!Files.exists(root) || !Files.isDirectory(root)) {
throw new CustomApiException(
ApiResponseCode.NOT_FOUND_DATA.getId(),
HttpStatus.CONFLICT,
"압축 해제 폴더가 존재하지 않습니다: " + unzipRootDir);
}
// 필요한 폴더들 존재/디렉토리 여부 확인
List<String> missing =
REQUIRED_DIRS.stream()
.filter(d -> !Files.isDirectory(root.resolve(d)))
.collect(Collectors.toList());
if (!missing.isEmpty()) {
throw new CustomApiException(
ApiResponseCode.NOT_FOUND_DATA.getId(),
HttpStatus.CONFLICT,
"데이터 폴더 구조가 올바르지 않습니다. 누락된 폴더: "
+ String.join(", ", missing)
+ " (필수: train, val, test)");
}
}
public static void validateDirFileCount(String unzipRootDir) {
Path root = Paths.get(unzipRootDir);
for (String split : REQUIRED_DIRS) {
Path splitPath = root.resolve(split);
Map<String, Long> fileCountMap = new HashMap<>();
for (String subDir : CHECK_DIRS) { // input1, input2, label 폴더만 수행하기
Path subDirPath = splitPath.resolve(subDir);
if (!Files.isDirectory(subDirPath)) {
throw new CustomApiException(
ApiResponseCode.NOT_FOUND_DATA.getId(),
HttpStatus.CONFLICT,
split + " 폴더 하위에 " + subDir + " 폴더가 존재하지 않습니다.");
}
long count;
try (Stream<Path> files = Files.list(subDirPath)) {
count = files.filter(Files::isRegularFile).count();
log.info("dir: " + subDirPath + ", count: " + count);
} catch (IOException e) {
throw new CustomApiException(
ApiResponseCode.NOT_FOUND_DATA.getId(),
HttpStatus.CONFLICT,
split + "/" + subDir + " 파일 개수 확인 중 오류 발생");
}
fileCountMap.put(subDir, count);
}
// 모든 폴더 파일 개수가 동일한지 확인
Set<Long> uniqueCounts = new HashSet<>(fileCountMap.values());
if (uniqueCounts.size() != 1) {
throw new CustomApiException(
ApiResponseCode.NOT_FOUND_DATA.getId(),
HttpStatus.CONFLICT,
split + " 데이터 파일 개수가 일치하지 않습니다. " + fileCountMap.toString());
}
}
}
}

View File

@@ -1,10 +1,11 @@
package com.kamco.cd.training.hyperparam;
import com.kamco.cd.training.common.dto.HyperParam;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.config.api.ApiResponseDto;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto.List;
import com.kamco.cd.training.hyperparam.service.HyperParamService;
import com.kamco.cd.training.model.dto.ModelMngDto;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.Parameter;
import io.swagger.v3.oas.annotations.media.Content;
@@ -49,8 +50,7 @@ public class HyperParamApiController {
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@PostMapping
public ApiResponseDto<String> createHyperParam(
@Valid @RequestBody HyperParamDto.HyperParamCreateReq createReq) {
public ApiResponseDto<String> createHyperParam(@Valid @RequestBody HyperParam createReq) {
String newVersion = hyperParamService.createHyperParam(createReq);
return ApiResponseDto.ok(newVersion);
}
@@ -66,11 +66,12 @@ public class HyperParamApiController {
mediaType = "application/json",
schema = @Schema(implementation = String.class))),
@ApiResponse(responseCode = "400", description = "잘못된 요청", content = @Content),
@ApiResponse(responseCode = "422", description = "default는 삭제불가", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@PutMapping("/{uuid}")
public ApiResponseDto<String> updateHyperParam(
@PathVariable UUID uuid, @Valid @RequestBody HyperParamDto.HyperParamCreateReq createReq) {
@PathVariable UUID uuid, @Valid @RequestBody HyperParam createReq) {
return ApiResponseDto.ok(hyperParamService.updateHyperParam(uuid, createReq));
}
@@ -96,10 +97,13 @@ public class HyperParamApiController {
String type,
@Parameter(description = "시작일", example = "2026-02-01") @RequestParam(required = false)
LocalDate startDate,
@Parameter(description = "종료일", example = "2026-02-28") @RequestParam(required = false)
@Parameter(description = "종료일", example = "2026-03-31") @RequestParam(required = false)
LocalDate endDate,
@Parameter(description = "버전명", example = "HPs_0001") @RequestParam(required = false)
@Parameter(description = "버전명", example = "G1_000019") @RequestParam(required = false)
String hyperVer,
@Parameter(description = "모델 타입 (G1, G2, G3 중 하나)", example = "G1")
@RequestParam(required = false)
ModelType model,
@Parameter(
description = "정렬",
example = "createdDttm desc",
@@ -124,7 +128,7 @@ public class HyperParamApiController {
searchReq.setSort(sort);
searchReq.setPage(page);
searchReq.setSize(size);
Page<List> list = hyperParamService.getHyperParamList(searchReq);
Page<List> list = hyperParamService.getHyperParamList(model, searchReq);
return ApiResponseDto.ok(list);
}
@@ -133,12 +137,12 @@ public class HyperParamApiController {
@ApiResponses(
value = {
@ApiResponse(responseCode = "200", description = "삭제 성공", content = @Content),
@ApiResponse(responseCode = "409", description = "HPs_0001 삭제 불가", content = @Content),
@ApiResponse(responseCode = "422", description = "default 삭제 불가", content = @Content),
@ApiResponse(responseCode = "404", description = "하이퍼파라미터를 찾을 수 없음", content = @Content),
})
@DeleteMapping("/{uuid}")
public ApiResponseDto<Void> deleteHyperParam(
@Parameter(description = "하이퍼파라미터 uuid", example = "7966dd64-004a-4596-89ef-001664bc4de2")
@Parameter(description = "하이퍼파라미터 uuid", example = "57fc9170-64c1-4128-aa7b-0657f08d6d10")
@PathVariable
UUID uuid) {
hyperParamService.deleteHyperParam(uuid);
@@ -154,13 +158,13 @@ public class HyperParamApiController {
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = ModelMngDto.HyperParamInfo.class))),
schema = @Schema(implementation = HyperParamDto.Basic.class))),
@ApiResponse(responseCode = "404", description = "하이퍼파라미터를 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/{uuid}")
public ApiResponseDto<HyperParamDto.Basic> getHyperParam(
@Parameter(description = "하이퍼파라미터 uuid", example = "9c91a20c-71e7-4e5f-a860-9626d2b2059c")
@Parameter(description = "하이퍼파라미터 uuid", example = "57fc9170-64c1-4128-aa7b-0657f08d6d10")
@PathVariable
UUID uuid) {
return ApiResponseDto.ok(hyperParamService.getHyperParam(uuid));
@@ -179,8 +183,9 @@ public class HyperParamApiController {
@ApiResponse(responseCode = "404", description = "하이퍼파라미터를 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/init")
public ApiResponseDto<HyperParamDto.Basic> getInitHyperParam() {
return ApiResponseDto.ok(hyperParamService.getInitHyperParam());
@GetMapping("/init/{model}")
public ApiResponseDto<HyperParamDto.Basic> getInitHyperParam(@PathVariable ModelType model) {
return ApiResponseDto.ok(hyperParamService.getInitHyperParam(model));
}
}

View File

@@ -1,5 +1,6 @@
package com.kamco.cd.training.hyperparam.dto;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.common.utils.enums.CodeExpose;
import com.kamco.cd.training.common.utils.enums.EnumType;
import com.kamco.cd.training.common.utils.interfaces.JsonFormatDttm;
@@ -24,7 +25,12 @@ public class HyperParamDto {
@AllArgsConstructor
public static class Basic {
private ModelType model; // 20250212 modeltype추가
private UUID uuid;
private String hyperVer;
@JsonFormatDttm private ZonedDateTime createdDttm;
@JsonFormatDttm private ZonedDateTime lastUsedDttm;
private Integer totalUseCnt;
// -------------------------
// Important
@@ -96,6 +102,8 @@ public class HyperParamDto {
private Integer gpuCnt;
private String gpuIds;
private Integer masterPort;
private Boolean isDefault;
}
@Getter
@@ -104,162 +112,12 @@ public class HyperParamDto {
@AllArgsConstructor
public static class List {
private UUID uuid;
private ModelType model;
private String hyperVer;
@JsonFormatDttm private ZonedDateTime createDttm;
@JsonFormatDttm private ZonedDateTime lastUsedDttm;
private Long m1UseCnt;
private Long m2UseCnt;
private Long m3UseCnt;
private Long totalCnt;
}
@Schema(name = "HyperParamCreateReq", description = "하이퍼파라미터 등록 요청")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class HyperParamCreateReq {
// -------------------------
// Important
// -------------------------
@Schema(description = "백본 네트워크", example = "large")
private String backbone; // backbone
@Schema(description = "입력 이미지 크기(H,W)", example = "512,512")
private String inputSize; // input_size
@Schema(description = "크롭 크기(H,W 또는 단일값)", example = "256,256")
private String cropSize; // crop_size
@Schema(description = "배치 크기(Per GPU)", example = "16")
private Integer batchSize; // batch_size
// -------------------------
// Data
// -------------------------
@Schema(description = "Train dataloader workers", example = "16")
private Integer trainNumWorkers; // train_num_workers
@Schema(description = "Val dataloader workers", example = "8")
private Integer valNumWorkers; // val_num_workers
@Schema(description = "Test dataloader workers", example = "8")
private Integer testNumWorkers; // test_num_workers
@Schema(description = "Train shuffle 여부", example = "true")
private Boolean trainShuffle; // train_shuffle
@Schema(description = "Train persistent workers 여부", example = "true")
private Boolean trainPersistent; // train_persistent
@Schema(description = "Val persistent workers 여부", example = "true")
private Boolean valPersistent; // val_persistent
// -------------------------
// Model Architecture
// -------------------------
@Schema(description = "Drop Path 비율", example = "0.3")
private Double dropPathRate; // drop_path_rate
@Schema(description = "Freeze 단계(-1:None)", example = "-1")
private Integer frozenStages; // frozen_stages
@Schema(description = "Neck 결합 정책", example = "abs_diff")
private String neckPolicy; // neck_policy
@Schema(description = "디코더 채널 구성", example = "512,256,128,64")
private String decoderChannels; // decoder_channels
@Schema(description = "클래스별 가중치", example = "1,10")
private String classWeight; // class_weight
// -------------------------
// Loss & Optimization
// -------------------------
@Schema(description = "학습률", example = "0.00006")
private Double learningRate; // learning_rate
@Schema(description = "Weight Decay", example = "0.05")
private Double weightDecay; // weight_decay
@Schema(description = "Layer Decay Rate", example = "0.9")
private Double layerDecayRate; // layer_decay_rate
@Schema(description = "DDP unused params 탐색 여부", example = "true")
private Boolean ddpFindUnusedParams; // ddp_find_unused_params
@Schema(description = "Loss 계산 제외 인덱스", example = "255")
private Integer ignoreIndex; // ignore_index
@Schema(description = "레이어 깊이", example = "24")
private Integer numLayers; // num_layers
// -------------------------
// Evaluation
// -------------------------
@Schema(description = "평가 지표 목록", example = "mFscore,mIoU")
private String metrics; // metrics
@Schema(description = "Best 모델 선정 기준 지표", example = "changed_fscore")
private String saveBest; // save_best
@Schema(description = "Best 모델 선정 규칙", example = "less")
private String saveBestRule; // save_best_rule
@Schema(description = "검증 수행 주기(Epoch)", example = "10")
private Integer valInterval; // val_interval
@Schema(description = "로그 기록 주기(Iteration)", example = "400")
private Integer logInterval; // log_interval
@Schema(description = "시각화 저장 주기(Epoch)", example = "1")
private Integer visInterval; // vis_interval
// -------------------------
// Augmentation
// -------------------------
@Schema(description = "회전 적용 확률", example = "0.5")
private Double rotProb; // rot_prob
@Schema(description = "회전 각도 범위(Min,Max)", example = "-20,20")
private String rotDegree; // rot_degree
@Schema(description = "반전 적용 확률", example = "0.5")
private Double flipProb; // flip_prob
@Schema(description = "채널 교환 확률", example = "0.5")
private Double exchangeProb; // exchange_prob
@Schema(description = "밝기 변화량", example = "10")
private Integer brightnessDelta; // brightness_delta
@Schema(description = "대비 범위(Min,Max)", example = "0.8,1.2")
private String contrastRange; // contrast_range
@Schema(description = "채도 범위(Min,Max)", example = "0.8,1.2")
private String saturationRange; // saturation_range
@Schema(description = "색조 변화량", example = "10")
private Integer hueDelta; // hue_delta
// -------------------------
// Hardware
// -------------------------
@Schema(description = "사용 GPU 개수", example = "4")
private Integer gpuCnt; // gpu_cnt
@Schema(description = "사용 GPU ID 목록", example = "0,1,2,3")
private String gpuIds; // gpu_ids
@Schema(description = "분산학습 마스터 포트", example = "1122")
private Integer masterPort; // master_port
// -------------------------
// Memo
// -------------------------
@Schema(description = "메모", example = "하이퍼파라미터 신규등록")
private String memo; // memo
private String memo;
private Integer totalUseCnt;
}
@Getter

View File

@@ -1,7 +1,10 @@
package com.kamco.cd.training.hyperparam.service;
import com.kamco.cd.training.common.dto.HyperParam;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto.List;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto.SearchReq;
import com.kamco.cd.training.postgres.core.HyperParamCoreService;
import java.util.UUID;
import lombok.RequiredArgsConstructor;
@@ -19,11 +22,12 @@ public class HyperParamService {
/**
* 하이퍼 파라미터 목록 조회
*
* @param model
* @param req
* @return 목록
*/
public Page<List> getHyperParamList(HyperParamDto.SearchReq req) {
return hyperParamCoreService.findByHyperVerList(req);
public Page<List> getHyperParamList(ModelType model, SearchReq req) {
return hyperParamCoreService.findByHyperVerList(model, req);
}
/**
@@ -33,8 +37,8 @@ public class HyperParamService {
* @return 생성된 버전명
*/
@Transactional
public String createHyperParam(HyperParamDto.HyperParamCreateReq createReq) {
return hyperParamCoreService.createHyperParam(createReq);
public String createHyperParam(HyperParam createReq) {
return hyperParamCoreService.createHyperParam(createReq).getHyperVer();
}
/**
@@ -44,7 +48,7 @@ public class HyperParamService {
* @return
*/
@Transactional
public String updateHyperParam(UUID uuid, HyperParamDto.HyperParamCreateReq createReq) {
public String updateHyperParam(UUID uuid, HyperParam createReq) {
return hyperParamCoreService.updateHyperParam(uuid, createReq);
}
@@ -58,8 +62,8 @@ public class HyperParamService {
}
/** 하이퍼파라미터 최적화 설정값 조회 */
public HyperParamDto.Basic getInitHyperParam() {
return hyperParamCoreService.getInitHyperParam();
public HyperParamDto.Basic getInitHyperParam(ModelType model) {
return hyperParamCoreService.getInitHyperParam(model);
}
/**

View File

@@ -0,0 +1,99 @@
package com.kamco.cd.training.log;
import com.kamco.cd.training.config.api.ApiResponseDto;
import com.kamco.cd.training.log.dto.AuditLogDto;
import com.kamco.cd.training.log.service.AuditLogService;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import java.time.LocalDate;
import lombok.RequiredArgsConstructor;
import org.springframework.data.domain.Page;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
@Tag(name = "감사 로그", description = "감사 로그 관리 API")
@RequiredArgsConstructor
@RestController
@RequestMapping("/api/logs/audit")
public class AuditLogApiController {
private final AuditLogService auditLogService;
@Operation(summary = "일자별 로그 조회")
@GetMapping("/daily")
public ApiResponseDto<Page<AuditLogDto.DailyAuditList>> getDailyLogs(
@RequestParam(required = false) LocalDate startDate,
@RequestParam(required = false) LocalDate endDate,
@RequestParam int page,
@RequestParam(defaultValue = "20") int size) {
AuditLogDto.searchReq searchReq = new AuditLogDto.searchReq(page, size, "created_dttm,desc");
Page<AuditLogDto.DailyAuditList> result =
auditLogService.getLogByDaily(searchReq, startDate, endDate);
return ApiResponseDto.ok(result);
}
@Operation(summary = "일자별 로그 상세")
@GetMapping("/daily/result")
public ApiResponseDto<Page<AuditLogDto.DailyDetail>> getDailyResultLogs(
@RequestParam LocalDate logDate,
@RequestParam int page,
@RequestParam(defaultValue = "20") int size) {
AuditLogDto.searchReq searchReq = new AuditLogDto.searchReq(page, size, "created_dttm,desc");
Page<AuditLogDto.DailyDetail> result = auditLogService.getLogByDailyResult(searchReq, logDate);
return ApiResponseDto.ok(result);
}
@Operation(summary = "메뉴별 로그 조회")
@GetMapping("/menu")
public ApiResponseDto<Page<AuditLogDto.MenuAuditList>> getMenuLogs(
@RequestParam(required = false) String searchValue,
@RequestParam int page,
@RequestParam(defaultValue = "20") int size) {
AuditLogDto.searchReq searchReq = new AuditLogDto.searchReq(page, size, "created_dttm,desc");
Page<AuditLogDto.MenuAuditList> result = auditLogService.getLogByMenu(searchReq, searchValue);
return ApiResponseDto.ok(result);
}
@Operation(summary = "메뉴별 로그 상세")
@GetMapping("/menu/result")
public ApiResponseDto<Page<AuditLogDto.MenuDetail>> getMenuResultLogs(
@RequestParam String menuId,
@RequestParam int page,
@RequestParam(defaultValue = "20") int size) {
AuditLogDto.searchReq searchReq = new AuditLogDto.searchReq(page, size, "created_dttm,desc");
Page<AuditLogDto.MenuDetail> result = auditLogService.getLogByMenuResult(searchReq, menuId);
return ApiResponseDto.ok(result);
}
@Operation(summary = "사용자별 로그 조회")
@GetMapping("/account")
public ApiResponseDto<Page<AuditLogDto.UserAuditList>> getAccountLogs(
@RequestParam(required = false) String searchValue,
@RequestParam int page,
@RequestParam(defaultValue = "20") int size) {
AuditLogDto.searchReq searchReq = new AuditLogDto.searchReq(page, size, "created_dttm,desc");
Page<AuditLogDto.UserAuditList> result =
auditLogService.getLogByAccount(searchReq, searchValue);
return ApiResponseDto.ok(result);
}
@Operation(summary = "사용자별 로그 상세")
@GetMapping("/account/result")
public ApiResponseDto<Page<AuditLogDto.UserDetail>> getAccountResultLogs(
@RequestParam Long userUid,
@RequestParam int page,
@RequestParam(defaultValue = "20") int size) {
AuditLogDto.searchReq searchReq = new AuditLogDto.searchReq(page, size, "created_dttm,desc");
Page<AuditLogDto.UserDetail> result = auditLogService.getLogByAccountResult(searchReq, userUid);
return ApiResponseDto.ok(result);
}
}

View File

@@ -0,0 +1,40 @@
package com.kamco.cd.training.log;
import com.kamco.cd.training.config.api.ApiResponseDto;
import com.kamco.cd.training.log.dto.ErrorLogDto;
import com.kamco.cd.training.log.dto.EventType;
import com.kamco.cd.training.log.service.ErrorLogService;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import java.time.LocalDate;
import lombok.RequiredArgsConstructor;
import org.springframework.data.domain.Page;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
@Tag(name = "에러 로그", description = "에러 로그 관리 API")
@RequiredArgsConstructor
@RestController
@RequestMapping({"/api/logs/system"})
public class ErrorLogApiController {
private final ErrorLogService errorLogService;
@Operation(summary = "에러로그 조회")
@GetMapping("/error")
public ApiResponseDto<Page<ErrorLogDto.Basic>> getErrorLogs(
@RequestParam(required = false) ErrorLogDto.LogErrorLevel logErrorLevel,
@RequestParam(required = false) EventType eventType,
@RequestParam(required = false) LocalDate startDate,
@RequestParam(required = false) LocalDate endDate,
@RequestParam int page,
@RequestParam(defaultValue = "20") int size) {
ErrorLogDto.ErrorSearchReq searchReq =
new ErrorLogDto.ErrorSearchReq(
logErrorLevel, eventType, startDate, endDate, page, size, "created_dttm,desc");
Page<ErrorLogDto.Basic> result = errorLogService.findLogByError(searchReq);
return ApiResponseDto.ok(result);
}
}

View File

@@ -3,7 +3,9 @@ package com.kamco.cd.training.log.dto;
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.kamco.cd.training.common.utils.interfaces.JsonFormatDttm;
import io.swagger.v3.oas.annotations.media.Schema;
import java.time.LocalDate;
import java.time.ZonedDateTime;
import java.util.UUID;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
@@ -58,6 +60,7 @@ public class AuditLogDto {
@Getter
@AllArgsConstructor
public static class AuditCommon {
private int readCount;
private int cudCount;
private int printCount;
@@ -68,6 +71,7 @@ public class AuditLogDto {
@Schema(name = "DailyAuditList", description = "일자별 목록")
@Getter
public static class DailyAuditList extends AuditCommon {
private final String baseDate;
public DailyAuditList(
@@ -85,6 +89,7 @@ public class AuditLogDto {
@Schema(name = "MenuAuditList", description = "메뉴별 목록")
@Getter
public static class MenuAuditList extends AuditCommon {
private final String menuId;
private final String menuName;
@@ -105,6 +110,7 @@ public class AuditLogDto {
@Schema(name = "UserAuditList", description = "사용자별 목록")
@Getter
public static class UserAuditList extends AuditCommon {
private final Long accountId;
private final String loginId;
private final String username;
@@ -129,6 +135,7 @@ public class AuditLogDto {
@Getter
@AllArgsConstructor
public static class AuditDetail {
private Long logId;
private EventType eventType;
private LogDetail detail;
@@ -137,9 +144,11 @@ public class AuditLogDto {
@Schema(name = "DailyDetail", description = "일자별 로그 상세")
@Getter
public static class DailyDetail extends AuditDetail {
private final String userName;
private final String loginId;
private final String menuName;
private final String logDateTime;
public DailyDetail(
Long logId,
@@ -147,17 +156,20 @@ public class AuditLogDto {
String loginId,
String menuName,
EventType eventType,
String logDateTime,
LogDetail detail) {
super(logId, eventType, detail);
this.userName = userName;
this.loginId = loginId;
this.menuName = menuName;
this.logDateTime = logDateTime;
}
}
@Schema(name = "MenuDetail", description = "메뉴별 로그 상세")
@Getter
public static class MenuDetail extends AuditDetail {
private final String logDateTime;
private final String userName;
private final String loginId;
@@ -179,6 +191,7 @@ public class AuditLogDto {
@Schema(name = "UserDetail", description = "사용자별 로그 상세")
@Getter
public static class UserDetail extends AuditDetail {
private final String logDateTime;
private final String menuNm;
@@ -194,6 +207,7 @@ public class AuditLogDto {
@Setter
@AllArgsConstructor
public static class LogDetail {
String serviceName;
String parentMenuName;
String menuName;
@@ -226,4 +240,26 @@ public class AuditLogDto {
return PageRequest.of(page, size);
}
}
@Getter
@Setter
public static class DownloadReq {
UUID uuid;
LocalDate startDate;
LocalDate endDate;
String searchValue;
String menuId;
String requestUri;
}
@Getter
@Setter
@AllArgsConstructor
public static class DownloadRes {
String name;
String employeeNo;
@JsonFormatDttm ZonedDateTime downloadDttm;
}
}

View File

@@ -1,5 +1,6 @@
package com.kamco.cd.training.log.dto;
import com.kamco.cd.training.common.utils.enums.CodeExpose;
import com.kamco.cd.training.common.utils.enums.EnumType;
import io.swagger.v3.oas.annotations.media.Schema;
import java.time.LocalDate;
@@ -77,6 +78,7 @@ public class ErrorLogDto {
}
}
@CodeExpose
public enum LogErrorLevel implements EnumType {
WARNING("Warning"),
ERROR("Error"),

View File

@@ -1,22 +1,35 @@
package com.kamco.cd.training.log.dto;
import com.kamco.cd.training.common.utils.enums.CodeExpose;
import com.kamco.cd.training.common.utils.enums.EnumType;
import lombok.AllArgsConstructor;
import lombok.Getter;
@CodeExpose
@Getter
@AllArgsConstructor
public enum EventType implements EnumType {
CREATE("생성"),
READ("조회"),
UPDATE("수정"),
DELETE("삭제"),
LIST("목록"),
DETAIL("상세"),
POPUP("팝업"),
STATUS("상태"),
ADDED("추가"),
MODIFIED("수정"),
REMOVE("삭제"),
DOWNLOAD("다운로드"),
PRINT("출력"),
LOGIN("로그인"),
OTHER("기타");
private final String desc;
public static EventType fromName(String name) {
try {
return EventType.valueOf(name.toUpperCase());
} catch (Exception e) {
return OTHER;
}
}
@Override
public String getId() {
return name();

View File

@@ -1,231 +0,0 @@
package com.kamco.cd.training.model;
import com.kamco.cd.training.config.api.ApiResponseDto;
import com.kamco.cd.training.model.dto.ModelMngDto;
import com.kamco.cd.training.model.dto.ModelMngDto.Basic;
import com.kamco.cd.training.model.service.ModelMngService;
import com.kamco.cd.training.model.service.ModelTrainService;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.Parameter;
import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.responses.ApiResponses;
import io.swagger.v3.oas.annotations.tags.Tag;
import jakarta.validation.Valid;
import java.util.List;
import lombok.RequiredArgsConstructor;
import org.springframework.data.domain.Page;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
@RestController
@RequiredArgsConstructor
@Tag(name = "모델관리", description = "어드민 홈 > 모델학습관리 > 모델관리 > 목록")
@RequestMapping("/api/models")
public class ModelMngApiController {
private final ModelMngService modelMngService;
private final ModelTrainService modelTrainService;
@Operation(summary = "학습 모델 목록 조회", description = "학습 모델 목록을 조회합니다")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "검색 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = Page.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping
public ApiResponseDto<Page<Basic>> findByModels(
@Parameter(description = "상태 코드") @RequestParam(required = false) String status,
@Parameter(description = "페이지 번호") @RequestParam(defaultValue = "0") int page,
@Parameter(description = "페이지 크기") @RequestParam(defaultValue = "20") int size) {
ModelMngDto.SearchReq searchReq = new ModelMngDto.SearchReq(status, page, size);
return ApiResponseDto.ok(modelMngService.findByModels(searchReq));
}
@Operation(summary = "학습 모델 상세 조회", description = "학습 모델의 상세 정보를 UUID로 조회합니다")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = ModelMngDto.Detail.class))),
@ApiResponse(responseCode = "404", description = "모델을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/{uuid}")
public ApiResponseDto<ModelMngDto.Detail> getModelDetail(
@Parameter(description = "모델 UUID", example = "b7e99739-6736-45f9-a224-8161ecddf287")
@PathVariable
String uuid) {
return ApiResponseDto.ok(modelMngService.getModelDetailByUuid(uuid));
}
// ==================== 학습 모델학습관리 API (5종) ====================
@Operation(summary = "학습 모델 통합 조회", description = "학습 관리 화면에서 학습 이력 리스트와 현재 상태를 조회합니다")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = List.class))),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/train")
public ApiResponseDto<List<ModelMngDto.TrainListRes>> getTrainModelList() {
return ApiResponseDto.ok(modelTrainService.getTrainModelList());
}
@Operation(summary = "학습 설정 통합 조회", description = "학습 실행 팝업 구성에 필요한 모든 데이터를 한 번에 반환합니다")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = ModelMngDto.FormConfigRes.class))),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/train/form-config")
public ApiResponseDto<ModelMngDto.FormConfigRes> getFormConfig() {
return ApiResponseDto.ok(modelTrainService.getFormConfig());
}
@Operation(summary = "학습 시작", description = "모든 설정(Step 1~3)을 마치고 최종적으로 학습 프로세스를 시작합니다")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "학습 시작 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = ModelMngDto.TrainStartRes.class))),
@ApiResponse(responseCode = "400", description = "잘못된 요청", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@PostMapping("/train")
public ApiResponseDto<ModelMngDto.TrainStartRes> startTraining(
@Valid @RequestBody ModelMngDto.TrainStartReq trainReq) {
return ApiResponseDto.ok(modelTrainService.startTraining(trainReq));
}
@Operation(summary = "학습 모델 삭제", description = "목록에서 특정 학습 모델을 삭제합니다")
@ApiResponses(
value = {
@ApiResponse(responseCode = "200", description = "삭제 성공", content = @Content),
@ApiResponse(responseCode = "400", description = "진행 중인 모델은 삭제 불가", content = @Content),
@ApiResponse(responseCode = "404", description = "모델을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@DeleteMapping("/train/{uuid}")
public ApiResponseDto<Void> deleteTrainModel(
@Parameter(description = "모델 UUID") @PathVariable String uuid) {
modelTrainService.deleteTrainModel(uuid);
return ApiResponseDto.ok(null);
}
// ==================== Resume Training (학습 재시작) ====================
@Operation(summary = "학습 재시작 정보 조회", description = "중단된 학습의 재시작 가능 여부와 Checkpoint 정보를 조회합니다")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = ModelMngDto.ResumeInfo.class))),
@ApiResponse(responseCode = "404", description = "모델을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/train/{uuid}/resume-info")
public ApiResponseDto<ModelMngDto.ResumeInfo> getResumeInfo(
@Parameter(description = "모델 UUID") @PathVariable String uuid) {
return ApiResponseDto.ok(modelTrainService.getResumeInfo(uuid));
}
@Operation(summary = "학습 재시작", description = "중단된 지점(Checkpoint)부터 학습을 재개합니다")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "재시작 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = ModelMngDto.ResumeResponse.class))),
@ApiResponse(responseCode = "400", description = "재시작 불가능한 상태", content = @Content),
@ApiResponse(responseCode = "404", description = "모델을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@PostMapping("/train/{uuid}/resume")
public ApiResponseDto<ModelMngDto.ResumeResponse> resumeTraining(
@Parameter(description = "모델 UUID") @PathVariable String uuid,
@Valid @RequestBody ModelMngDto.ResumeRequest resumeReq) {
return ApiResponseDto.ok(modelTrainService.resumeTraining(uuid, resumeReq));
}
// ==================== Best Epoch Setting (Best Epoch 설정) ====================
@Operation(summary = "Best Epoch 설정", description = "사용자가 직접 Best Epoch를 선택하여 설정합니다")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "설정 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = ModelMngDto.BestEpochResponse.class))),
@ApiResponse(responseCode = "404", description = "모델을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@PostMapping("/train/{uuid}/best-epoch")
public ApiResponseDto<ModelMngDto.BestEpochResponse> setBestEpoch(
@Parameter(description = "모델 UUID") @PathVariable String uuid,
@Valid @RequestBody ModelMngDto.BestEpochRequest bestEpochReq) {
return ApiResponseDto.ok(modelTrainService.setBestEpoch(uuid, bestEpochReq));
}
@Operation(summary = "Epoch별 성능 지표 조회", description = "학습된 모델의 Epoch별 성능 지표를 조회합니다")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = List.class))),
@ApiResponse(responseCode = "404", description = "모델을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/train/{uuid}/epoch-metrics")
public ApiResponseDto<List<ModelMngDto.EpochMetric>> getEpochMetrics(
@Parameter(description = "모델 UUID") @PathVariable String uuid) {
return ApiResponseDto.ok(modelTrainService.getEpochMetrics(uuid));
}
}

View File

@@ -0,0 +1,329 @@
package com.kamco.cd.training.model;
import com.kamco.cd.training.common.download.RangeDownloadResponder;
import com.kamco.cd.training.config.api.ApiResponseDto;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.MappingDataset;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelBestEpoch;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelFileInfo;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelTestMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelTrainMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelValidationMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.TransferDetailDto;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.Basic;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.ModelProgressStepDto;
import com.kamco.cd.training.model.service.ModelTrainDetailService;
import com.kamco.cd.training.model.service.ModelTrainMngService;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.Parameter;
import io.swagger.v3.oas.annotations.enums.ParameterIn;
import io.swagger.v3.oas.annotations.media.ArraySchema;
import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.responses.ApiResponses;
import io.swagger.v3.oas.annotations.tags.Tag;
import jakarta.servlet.http.HttpServletRequest;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.List;
import java.util.UUID;
import lombok.RequiredArgsConstructor;
import org.apache.coyote.BadRequestException;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
@RequiredArgsConstructor
@Tag(name = "모델학습 관리", description = "어드민 홈 > 모델학습관리 > 모델관리 > 목록")
@RequestMapping("/api/models")
public class ModelTrainDetailApiController {
private final ModelTrainDetailService modelTrainDetailService;
private final ModelTrainMngService modelTrainMngService;
private final RangeDownloadResponder rangeDownloadResponder;
@Value("${train.docker.responseDir}")
private String responseDir;
@Operation(summary = "모델학습관리> 모델관리 > 상세정보탭 > 학습 진행정보", description = "학습 진행정보, 모델학습 정보 API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = Basic.class))),
@ApiResponse(responseCode = "404", description = "데이터셋을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/detail/{uuid}")
public ApiResponseDto<Basic> findByModelByUUID(
@Parameter(description = "모델학습 uuid", example = "7fbdff54-ea87-4b02-90d1-955fa2a3457e")
@PathVariable
UUID uuid) {
return ApiResponseDto.ok(modelTrainDetailService.findByModelByUUID(uuid));
}
@Operation(summary = "모델학습관리> 모델관리 > 상세정보탭 > 요약정보", description = "상세정보 탭 요약정보 API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = ModelTrainDetailDto.DetailSummary.class))),
@ApiResponse(responseCode = "404", description = "데이터셋을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/summary/{uuid}")
public ApiResponseDto<ModelTrainDetailDto.DetailSummary> getModelDetailSummary(
@Parameter(description = "모델학습 uuid", example = "7fbdff54-ea87-4b02-90d1-955fa2a3457e")
@PathVariable
UUID uuid) {
return ApiResponseDto.ok(modelTrainDetailService.getModelDetailSummary(uuid));
}
@Operation(summary = "모델학습관리> 모델관리 > 상세정보탭 > 하이퍼파라미터 요약 정보", description = "하이퍼파라미터 요약 정보 API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = ModelTrainDetailDto.HyperSummary.class))),
@ApiResponse(responseCode = "404", description = "데이터셋을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/hyper-summary/{uuid}")
public ApiResponseDto<ModelTrainDetailDto.HyperSummary> getByModelHyperParamSummary(
@Parameter(description = "모델학습 uuid", example = "7fbdff54-ea87-4b02-90d1-955fa2a3457e")
@PathVariable
UUID uuid) {
return ApiResponseDto.ok(modelTrainDetailService.getByModelHyperParamSummary(uuid));
}
@Operation(summary = "모델학습관리> 모델관리 > 상세정보탭 > 데이터셋 정보", description = "모델학습 상세 데이터셋 정보 API")
@ApiResponses({
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
array = @ArraySchema(schema = @Schema(implementation = MappingDataset.class)))),
@ApiResponse(responseCode = "404", description = "데이터셋을 찾을 수 없음"),
@ApiResponse(responseCode = "500", description = "서버 오류")
})
@GetMapping("/mapp-dataset/{uuid}")
public ApiResponseDto<List<MappingDataset>> getByModelMappingDataset(
@Parameter(description = "모델학습 uuid", example = "7fbdff54-ea87-4b02-90d1-955fa2a3457e")
@PathVariable
UUID uuid) {
return ApiResponseDto.ok(modelTrainDetailService.getByModelMappingDataset(uuid));
}
// @Operation(summary = "모델관리 > 전이 학습 실행설정 > 모델선택", description = "모델선택 정보 API")
// @ApiResponses(
// value = {
// @ApiResponse(
// responseCode = "200",
// description = "조회 성공",
// content =
// @Content(
// mediaType = "application/json",
// schema = @Schema(implementation = TransferDetailDto.class))),
// @ApiResponse(responseCode = "404", description = "데이터셋을 찾을 수 없음", content = @Content),
// @ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
// })
// @GetMapping("/transfer/detail/{uuid}")
// public ApiResponseDto<TransferDetailDto> getTransferDetail(
// @Parameter(description = "모델 uuid", example = "7fbdff54-ea87-4b02-90d1-955fa2a3457e")
// @PathVariable
// UUID uuid) {
// return ApiResponseDto.ok(modelTrainDetailService.getTransferDetail(uuid));
// }
@Operation(summary = "모델관리 > 모델 상세 > 성능 정보 (Train)", description = "모델 상세 > 성능 정보 (Train) API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = TransferDetailDto.class))),
@ApiResponse(responseCode = "404", description = "데이터셋을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/metrics/train/{uuid}")
public ApiResponseDto<List<ModelTrainMetrics>> getModelTrainMetricResult(
@Parameter(description = "모델 uuid", example = "95cb116c-380a-41c0-98d8-4d1142f15bbf")
@PathVariable
UUID uuid) {
return ApiResponseDto.ok(modelTrainDetailService.getModelTrainMetricResult(uuid));
}
@Operation(
summary = "모델관리 > 모델 상세 > 성능 정보 (Validation)",
description = "모델 상세 > 성능 정보 (Validation) API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = TransferDetailDto.class))),
@ApiResponse(responseCode = "404", description = "데이터셋을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/metrics/validation/{uuid}")
public ApiResponseDto<List<ModelValidationMetrics>> getModelValidationMetricResult(
@Parameter(description = "모델 uuid", example = "95cb116c-380a-41c0-98d8-4d1142f15bbf")
@PathVariable
UUID uuid) {
return ApiResponseDto.ok(modelTrainDetailService.getModelValidationMetricResult(uuid));
}
@Operation(summary = "모델관리 > 모델 상세 > 성능 정보 (Test)", description = "모델 상세 > 성능 정보 (Test) API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = TransferDetailDto.class))),
@ApiResponse(responseCode = "404", description = "데이터셋을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/metrics/test/{uuid}")
public ApiResponseDto<List<ModelTestMetrics>> getModelTestMetricResult(
@Parameter(description = "모델 uuid", example = "95cb116c-380a-41c0-98d8-4d1142f15bbf")
@PathVariable
UUID uuid) {
return ApiResponseDto.ok(modelTrainDetailService.getModelTestMetricResult(uuid));
}
@Operation(summary = "모델관리 > 모델 상세 > 성능 정보 (Test)", description = "모델 상세 > 성능 정보 (Test) API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = TransferDetailDto.class))),
@ApiResponse(responseCode = "404", description = "데이터셋을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/best-epoch/{uuid}")
public ApiResponseDto<ModelBestEpoch> getModelTrainBestEpoch(
@Parameter(description = "모델 uuid", example = "95cb116c-380a-41c0-98d8-4d1142f15bbf")
@PathVariable
UUID uuid) {
return ApiResponseDto.ok(modelTrainDetailService.getModelTrainBestEpoch(uuid));
}
@Operation(
summary = "학습데이터 파일 다운로드",
description = "학습데이터 파일 다운로드",
parameters = {
@Parameter(
name = "kamco-download-uuid",
in = ParameterIn.HEADER,
required = true,
description = "다운로드 요청 UUID",
schema =
@Schema(
type = "string",
format = "uuid",
example = "6d8d49dc-0c9d-4124-adc7-b9ca610cc394"))
})
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "학습데이터 zip파일 다운로드",
content =
@Content(
mediaType = "application/octet-stream",
schema = @Schema(type = "string", format = "binary"))),
@ApiResponse(responseCode = "404", description = "파일 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/download/{uuid}")
public ResponseEntity<?> download(@PathVariable UUID uuid, HttpServletRequest request)
throws IOException {
Basic info = modelTrainDetailService.findByModelByUUID(uuid);
Path zipPath =
Paths.get(responseDir)
.resolve(String.valueOf(info.getUuid()))
.resolve(info.getModelVer() + ".zip");
if (!Files.isRegularFile(zipPath)) {
throw new BadRequestException();
}
return rangeDownloadResponder.buildZipResponse(zipPath, info.getModelVer() + ".zip", request);
}
@Operation(summary = "모델관리 > 모델 상세 > 파일 정보", description = "모델 상세 > 파일 정보 API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = TransferDetailDto.class))),
@ApiResponse(responseCode = "404", description = "데이터셋을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/file-info/{uuid}")
public ApiResponseDto<ModelFileInfo> getModelTrainFileInfo(
@Parameter(description = "모델 uuid", example = "95cb116c-380a-41c0-98d8-4d1142f15bbf")
@PathVariable
UUID uuid) {
return ApiResponseDto.ok(modelTrainDetailService.getModelTrainFileInfo(uuid));
}
@Operation(summary = "모델관리 > 모델별 진행 상황", description = "모델관리 > 모델별 진행 상황 API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = ModelProgressStepDto.class))),
@ApiResponse(responseCode = "404", description = "데이터셋을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/progress/{uuid}")
public ApiResponseDto<List<ModelProgressStepDto>> findModelTrainProgressInfo(
@Parameter(description = "모델 uuid", example = "95cb116c-380a-41c0-98d8-4d1142f15bbf")
@PathVariable
UUID uuid) {
return ApiResponseDto.ok(modelTrainDetailService.findModelTrainProgressInfo(uuid));
}
}

View File

@@ -0,0 +1,217 @@
package com.kamco.cd.training.model;
import com.kamco.cd.training.config.api.ApiResponseDto;
import com.kamco.cd.training.dataset.dto.DatasetDto;
import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetReq;
import com.kamco.cd.training.dataset.dto.DatasetDto.SelectDataSet;
import com.kamco.cd.training.model.dto.ModelConfigDto;
import com.kamco.cd.training.model.dto.ModelTrainMngDto;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.ListDto;
import com.kamco.cd.training.model.service.ModelTrainMngService;
import com.kamco.cd.training.train.service.ModelTestMetricsJobService;
import com.kamco.cd.training.train.service.ModelTrainMetricsJobService;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.Parameter;
import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.responses.ApiResponses;
import io.swagger.v3.oas.annotations.tags.Tag;
import jakarta.validation.Valid;
import java.io.IOException;
import java.util.List;
import java.util.UUID;
import lombok.RequiredArgsConstructor;
import org.springframework.data.domain.Page;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
@RestController
@RequiredArgsConstructor
@Tag(name = "모델학습 관리", description = "어드민 홈 > 모델학습관리 > 모델관리 > 목록")
@RequestMapping("/api/models")
public class ModelTrainMngApiController {
private final ModelTrainMngService modelTrainMngService;
private final ModelTrainMetricsJobService modelTrainMetricsJobService;
private final ModelTestMetricsJobService modelTestMetricsJobService;
@Operation(summary = "모델학습 목록 조회", description = "모델학습 목록 조회 API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "검색 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = Page.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/list")
public ApiResponseDto<Page<ListDto>> findByModelList(
@Parameter(
description = "상태코드",
example = "IN_PROGRESS",
schema = @Schema(allowableValues = {"", "IN_PROGRESS", "COMPLETED"}))
@RequestParam(required = false)
String status,
@Parameter(
description = "모델",
example = "G1",
schema = @Schema(allowableValues = {"G1", "G2", "G3"}))
@RequestParam(required = false)
String modelNo,
@Parameter(description = "페이지 번호") @RequestParam(defaultValue = "0") int page,
@Parameter(description = "페이지 크기") @RequestParam(defaultValue = "20") int size) {
ModelTrainMngDto.SearchReq searchReq =
new ModelTrainMngDto.SearchReq(status, modelNo, page, size);
return ApiResponseDto.ok(modelTrainMngService.getModelList(searchReq));
}
@Operation(summary = "학습 모델 삭제", description = "학습 모델 삭제 API")
@ApiResponses(
value = {
@ApiResponse(responseCode = "200", description = "삭제 성공", content = @Content),
@ApiResponse(responseCode = "409", description = "G1_000001 삭제 불가", content = @Content)
})
@DeleteMapping("/{uuid}")
public ApiResponseDto<Void> deleteModelTrain(
@Parameter(description = "학습 모델 uuid", example = "f2b02229-90f2-45f5-92ea-c56cf1c29f79")
@PathVariable
UUID uuid) {
modelTrainMngService.deleteModelTrain(uuid);
return ApiResponseDto.ok(null);
}
@Operation(summary = "학습 모델 등록", description = "학습 모델 등록 API")
@ApiResponses(
value = {
@ApiResponse(responseCode = "200", description = "등록 성공", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@PostMapping
public ApiResponseDto<UUID> createModelTrain(@Valid @RequestBody ModelTrainMngDto.AddReq req) {
return ApiResponseDto.ok(modelTrainMngService.createModelTrain(req));
}
@Operation(summary = "모델학습 config 정보 조회", description = "모델학습 config 정보 조회 API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "검색 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = ModelConfigDto.Basic.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/config/{uuid}")
public ApiResponseDto<ModelConfigDto.Basic> updateModelTrain(
@Parameter(description = "모델학습 uuid", example = "7fbdff54-ea87-4b02-90d1-955fa2a3457e")
@PathVariable
UUID uuid) {
return ApiResponseDto.ok(modelTrainMngService.getModelConfigByModelId(uuid));
}
@Operation(summary = "모델별 데이터셋 목록 조회", description = "모델별 데이터셋 목록 조회 API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = DatasetDto.Basic.class))),
@ApiResponse(responseCode = "404", description = "데이터셋을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/select-dataset-list")
public ApiResponseDto<List<SelectDataSet>> getDatasetSelectList(
@Parameter(
description = "모델 구분",
example = "",
schema = @Schema(allowableValues = {"G1", "G2", "G3"}))
@RequestParam
String modelType,
@Parameter(
description = "선택 구분",
example = "",
schema = @Schema(allowableValues = {"CURRENT", "DELIVER", "PRODUCTION"}))
@RequestParam
String selectType) {
DatasetReq req = new DatasetReq();
req.setModelNo(modelType);
req.setDataType(selectType);
return ApiResponseDto.ok(modelTrainMngService.getDatasetSelectList(req));
}
@Operation(
summary = "모델학습 1단계/2단계 실행중인 것이 있는지 count",
description = "모델학습 1단계/2단계 실행중인 것이 있는지 count")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "검색 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = Long.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/ing-training-cnt")
public ApiResponseDto<Long> findModelStep1InProgressCnt() {
return ApiResponseDto.ok(modelTrainMngService.findModelStep1InProgressCnt());
}
@Operation(
summary = "스케줄러 findTrainValidMetricCsvFiles",
description = "스케줄러 findTrainValidMetricCsvFiles")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "검색 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = Long.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/schedule-trainvalid")
public ApiResponseDto<Long> findTrainValidMetricCsvFiles() {
modelTrainMetricsJobService.findTrainValidMetricCsvFiles();
return ApiResponseDto.ok(null);
}
@Operation(summary = "스케줄러 findTestMetricCsvFiles", description = "스케줄러 findTestMetricCsvFiles")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "검색 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = Long.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/schedule-test")
public ApiResponseDto<Long> findTestValidMetricCsvFiles() throws IOException {
modelTestMetricsJobService.findTestValidMetricCsvFiles();
return ApiResponseDto.ok(null);
}
}

View File

@@ -0,0 +1,44 @@
package com.kamco.cd.training.model.dto;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
public class ModelConfigDto {
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class Basic {
private Long configId;
private Long modelId;
private Integer epochCount;
private Float trainPercent;
private Float validationPercent;
private Float testPercent;
private String memo;
}
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class TransferBasic {
private Long configId;
private Long modelId;
private Integer epochCount;
private Float trainPercent;
private Float validationPercent;
private Float testPercent;
private String memo;
private Long beforeConfigId;
private Long beforeModelId;
private Integer beforeEpochCount;
private Float beforeTrainPercent;
private Float beforeValidationPercent;
private Float beforeTestPercent;
private String beforeMemo;
}
}

View File

@@ -1,415 +0,0 @@
package com.kamco.cd.training.model.dto;
import com.kamco.cd.training.common.utils.interfaces.JsonFormatDttm;
import io.swagger.v3.oas.annotations.media.Schema;
import jakarta.validation.constraints.NotBlank;
import jakarta.validation.constraints.NotEmpty;
import jakarta.validation.constraints.NotNull;
import java.time.ZonedDateTime;
import java.util.List;
import java.util.Map;
import java.util.UUID;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import org.springframework.data.domain.PageRequest;
import org.springframework.data.domain.Pageable;
public class ModelMngDto {
@Schema(name = "모델관리 목록 조회", description = "모델관리 목록 조회")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class Basic {
private Long id;
private UUID uuid;
private String modelVer;
@JsonFormatDttm private ZonedDateTime startDttm;
@JsonFormatDttm private ZonedDateTime step1EndDttm;
@JsonFormatDttm private ZonedDateTime step1Duration;
@JsonFormatDttm private ZonedDateTime step2EndDttm;
@JsonFormatDttm private ZonedDateTime step2Duration;
private String step1Status;
private String step2Status;
private String transferStatus;
}
@Schema(name = "searchReq", description = "모델 관리 목록조회 파라미터")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class SearchReq {
private String status;
// 페이징 파라미터
private int page = 0;
private int size = 20;
public Pageable toPageable() {
return PageRequest.of(page, size);
}
}
@Schema(name = "Detail", description = "모델 상세 정보")
@Getter
@Builder
public static class Detail {
private String uuid;
private String modelVer;
private String hyperVer;
private String epochVer;
private String processStep;
private String statusCd;
private String statusText;
@JsonFormatDttm private ZonedDateTime trainStartDttm;
private Integer epochCnt;
private String datasetRatio;
private Integer bestEpoch;
private Integer confirmedBestEpoch;
@JsonFormatDttm private ZonedDateTime step1EndDttm;
private ZonedDateTime step1Duration;
@JsonFormatDttm private ZonedDateTime step2EndDttm;
private ZonedDateTime step2Duration;
private Integer progressRate;
@JsonFormatDttm private ZonedDateTime createdDttm;
@JsonFormatDttm private ZonedDateTime updatedDttm;
private String modelPath;
private String errorMsg;
}
@Schema(name = "TrainListRes", description = "학습 모델 목록 응답")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class TrainListRes {
private String uuid;
private String modelVer;
private String status;
private String processStep;
@JsonFormatDttm private ZonedDateTime trainStartDttm;
private Integer progressRate;
private Integer epochCnt;
@JsonFormatDttm private ZonedDateTime step1EndDttm;
private ZonedDateTime step1Duration;
@JsonFormatDttm private ZonedDateTime step2EndDttm;
private ZonedDateTime step2Duration;
@JsonFormatDttm private ZonedDateTime createdDttm;
private String errorMsg;
private Boolean canResume;
private Integer lastCheckpointEpoch;
}
@Schema(name = "FormConfigRes", description = "학습 설정 통합 조회 응답")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class FormConfigRes {
private Boolean isTrainAvailable;
private List<HyperParamInfo> hyperParams;
private List<DatasetInfo> datasets;
private String runningModelUuid;
}
@Schema(name = "HyperParamInfo", description = "하이퍼파라미터 정보")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class HyperParamInfo {
@Schema(description = "하이퍼파라미터 버전", example = "V3.99.251221.120518")
private String hyperVer;
// Important
@Schema(description = "백본", example = "large")
private String backbone;
@Schema(description = "입력 사이즈", example = "256,256")
private String inputSize;
@Schema(description = "크롭 사이즈", example = "256,256")
private String cropSize;
@Schema(description = "에폭 수", example = "200")
private Integer epochCnt;
@Schema(description = "배치 사이즈", example = "16")
private Integer batchSize;
// Architecture
@Schema(description = "Drop Path Rate", example = "0.3")
private Double dropPathRate;
@Schema(description = "Frozen Stages", example = "-1")
private Integer frozenStages;
@Schema(description = "Neck Policy", example = "abs_diff")
private String neckPolicy;
@Schema(description = "Decoder Channels", example = "512,256,128,64")
private String decoderChannels;
@Schema(description = "Class Weight", example = "1,1")
private String classWeight;
@Schema(description = "레이어 수", example = "24")
private Integer numLayers;
// Optimization
@Schema(description = "Learning Rate", example = "0.00006")
private Double learningRate;
@Schema(description = "Weight Decay", example = "0.05")
private Double weightDecay;
@Schema(description = "Layer Decay Rate", example = "0.9")
private Double layerDecayRate;
@Schema(description = "DDP Unused Params 찾기", example = "true")
private Boolean ddpFindUnusedParams;
@Schema(description = "Ignore Index", example = "255")
private Integer ignoreIndex;
// Data
@Schema(description = "Train Workers", example = "16")
private Integer trainNumWorkers;
@Schema(description = "Val Workers", example = "8")
private Integer valNumWorkers;
@Schema(description = "Test Workers", example = "8")
private Integer testNumWorkers;
@Schema(description = "Train Shuffle", example = "true")
private Boolean trainShuffle;
@Schema(description = "Train Persistent", example = "true")
private Boolean trainPersistent;
@Schema(description = "Val Persistent", example = "true")
private Boolean valPersistent;
// Evaluation
@Schema(description = "Metrics", example = "mFscore,mIoU")
private String metrics;
@Schema(description = "Save Best", example = "changed_fscore")
private String saveBest;
@Schema(description = "Save Best Rule", example = "greater")
private String saveBestRule;
@Schema(description = "Val Interval", example = "10")
private Integer valInterval;
@Schema(description = "Log Interval", example = "400")
private Integer logInterval;
@Schema(description = "Vis Interval", example = "1")
private Integer visInterval;
// Hardware
@Schema(description = "GPU 수", example = "4")
private Integer gpuCnt;
@Schema(description = "GPU IDs", example = "0,1,2,3")
private String gpuIds;
@Schema(description = "Master Port", example = "1122")
private Integer masterPort;
// Augmentation
@Schema(description = "Rotation 확률", example = "0.5")
private Double rotProb;
@Schema(description = "Flip 확률", example = "0.5")
private Double flipProb;
@Schema(description = "Rotation 각도", example = "-20,20")
private String rotDegree;
@Schema(description = "Exchange 확률", example = "0.5")
private Double exchangeProb;
@Schema(description = "Brightness Delta", example = "10")
private Integer brightnessDelta;
@Schema(description = "Contrast Range", example = "0.8,1.2")
private String contrastRange;
@Schema(description = "Saturation Range", example = "0.8,1.2")
private String saturationRange;
@Schema(description = "Hue Delta", example = "10")
private Integer hueDelta;
// Legacy
private Double dropoutRatio;
private Integer cnnFilterCnt;
// Common
@Schema(description = "메모", example = "안녕하세요 캠코담당자 입니다. 하이퍼파라미터 신규등록합니다")
private String memo;
@JsonFormatDttm private ZonedDateTime createdDttm;
}
@Schema(name = "DatasetInfo", description = "데이터셋 정보")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class DatasetInfo {
private Long id;
private String title;
private String groupTitle;
private Long totalItems;
private String totalSize;
private Map<String, Integer> classCounts;
private String memo;
@JsonFormatDttm private ZonedDateTime createdDttm;
}
@Schema(name = "TrainStartReq", description = "학습 시작 요청")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class TrainStartReq {
@NotBlank(message = "하이퍼파라미터 버전은 필수입니다")
@Schema(example = "V3.99.251221.120518")
private String hyperVer;
@NotEmpty(message = "데이터셋은 최소 1개 이상 선택해야 합니다")
private List<Long> datasetIds;
@NotNull(message = "에폭 수는 필수입니다")
@jakarta.validation.constraints.Min(value = 1, message = "에폭 수는 최소 1 이상이어야 합니다")
@jakarta.validation.constraints.Max(value = 200, message = "에폭 수는 최대 200까지 설정 가능합니다")
@Schema(example = "200")
private Integer epoch;
@Schema(example = "7:2:1", description = "데이터 분할 비율 (Training:Validation:Test)")
private String datasetRatio;
}
@Schema(name = "TrainStartRes", description = "학습 시작 응답")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class TrainStartRes {
private String uuid;
private String status;
}
@Schema(name = "ResumeInfo", description = "학습 재시작 정보")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class ResumeInfo {
private Boolean canResume;
private Integer lastEpoch;
private Integer totalEpoch;
private String checkpointPath;
@JsonFormatDttm private ZonedDateTime failedAt;
}
@Schema(name = "ResumeRequest", description = "학습 재시작 요청")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class ResumeRequest {
@NotNull(message = "재시작 Epoch는 필수입니다")
private Integer resumeFromEpoch;
private Integer newTotalEpoch;
}
@Schema(name = "ResumeResponse", description = "학습 재시작 응답")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class ResumeResponse {
private String uuid;
private String status;
private Integer resumedFromEpoch;
}
@Schema(name = "BestEpochRequest", description = "Best Epoch 설정 요청")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class BestEpochRequest {
@NotNull(message = "Best Epoch는 필수입니다")
private Integer bestEpoch;
private String reason;
}
@Schema(name = "BestEpochResponse", description = "Best Epoch 설정 응답")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class BestEpochResponse {
private String uuid;
private Integer bestEpoch;
private Integer confirmedBestEpoch;
private Integer previousBestEpoch;
}
@Schema(name = "EpochMetric", description = "Epoch별 성능 지표")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class EpochMetric {
private Integer epoch;
private Double mIoU;
private Double mFscore;
private Double loss;
private Boolean isBest;
}
}

View File

@@ -0,0 +1,259 @@
package com.kamco.cd.training.model.dto;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.kamco.cd.training.common.enums.LearnDataType;
import com.kamco.cd.training.common.enums.TrainStatusType;
import com.kamco.cd.training.common.enums.TrainType;
import com.kamco.cd.training.common.utils.enums.Enums;
import com.kamco.cd.training.common.utils.interfaces.JsonFormatDttm;
import com.kamco.cd.training.dataset.dto.DatasetDto.SelectTransferDataSet;
import io.swagger.v3.oas.annotations.media.Schema;
import java.time.Duration;
import java.time.ZonedDateTime;
import java.util.List;
import java.util.UUID;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
public class ModelTrainDetailDto {
@Schema(name = "모델학습관리 목록", description = "모델학습관리 목록")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class DetailSummary {
private Long modelId;
private UUID uuid;
private String modelNo;
private String modelVer;
@JsonFormatDttm private ZonedDateTime step1StrtDttm;
@JsonFormatDttm private ZonedDateTime step2EndDttm;
private String statusCd;
private String trainType;
private UUID beforeUuid;
public String getStatusName() {
if (this.statusCd == null || this.statusCd.isBlank()) return null;
try {
return TrainStatusType.valueOf(this.statusCd).getText(); // 또는 getName()
} catch (IllegalArgumentException e) {
return this.statusCd; // 매핑 못하면 코드 그대로 반환(원하면 null 처리)
}
}
public String getTrainTypeName() {
if (this.trainType == null || this.trainType.isBlank()) return null;
try {
return TrainType.valueOf(this.trainType).getText(); // 또는 getName()
} catch (IllegalArgumentException e) {
return this.trainType; // 매핑 못하면 코드 그대로 반환(원하면 null 처리)
}
}
private String formatDuration(ZonedDateTime start, ZonedDateTime end) {
if (end == null) {
end = ZonedDateTime.now();
}
if (start == null) {
return null;
}
long totalSeconds = Math.abs(Duration.between(start, end).getSeconds());
long hours = totalSeconds / 3600;
long minutes = (totalSeconds % 3600) / 60;
long seconds = totalSeconds % 60;
return String.format("%d시간 %d분 %d초", hours, minutes, seconds);
}
public String getStepAllDuration() {
return formatDuration(this.step1StrtDttm, this.step2EndDttm);
}
}
@Schema(name = "모델학습관리 목록", description = "모델학습관리 목록")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class HyperSummary {
private UUID uuid;
private Long hyperParamId;
private String hyperVer;
private String backbone;
private String inputSize;
private String cropSize;
private Integer batchSize;
}
@Schema(name = "모델학습관리 전이 하이파라미터", description = "모델학습관리 전이 하이파라미터")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class TransferHyperSummary {
private UUID uuid;
private Long hyperParamId;
private String hyperVer;
private String backbone;
private String inputSize;
private String cropSize;
private Integer batchSize;
private UUID beforeUuid;
private Long beforeHyperParamId;
private String beforeHyperVer;
private String beforeBackbone;
private String beforeInputSize;
private String beforeCropSize;
private Integer beforeBatchSize;
}
@Schema(name = "선택한 데이터셋 목록", description = "선택한 데이터셋 목록")
@Getter
@Setter
@NoArgsConstructor
public static class MappingDataset {
private Long modelId;
private Long datasetId;
private String dataType;
private Integer compareYyyy;
private Integer targetYyyy;
private Long roundNo;
private String dataTypeName;
@JsonInclude(JsonInclude.Include.NON_NULL)
private Long buildingCnt;
@JsonInclude(JsonInclude.Include.NON_NULL)
private Long containerCnt;
@JsonInclude(JsonInclude.Include.NON_NULL)
private Long wasteCnt;
@JsonInclude(JsonInclude.Include.NON_NULL)
private Long landCoverCnt;
public MappingDataset(
Long modelId,
Long datasetId,
String dataType,
Integer compareYyyy,
Integer targetYyyy,
Long roundNo,
Long buildingCnt,
Long containerCnt,
Long wasteCnt,
Long landCoverCnt) {
this.modelId = modelId;
this.datasetId = datasetId;
this.dataType = dataType;
this.compareYyyy = compareYyyy;
this.targetYyyy = targetYyyy;
this.roundNo = roundNo;
this.buildingCnt = buildingCnt;
this.containerCnt = containerCnt;
this.wasteCnt = wasteCnt;
this.landCoverCnt = landCoverCnt;
this.dataTypeName = getDataTypeName(this.dataType);
}
public String getDataTypeName(String groupTitleCd) {
LearnDataType type = Enums.fromId(LearnDataType.class, groupTitleCd);
return type == null ? null : type.getText();
}
}
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class TransferDetailDto {
private ModelConfigDto.TransferBasic etcConfig;
private TransferHyperSummary modelTrainHyper;
private List<SelectTransferDataSet> modelTrainDataset;
// private List<SelectDataSet> beforeTrainDataset;
}
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class ModelTrainMetrics {
private Integer epoch;
private Long iteration;
private Double loss;
private Double lr;
private Float durationTime;
}
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class ModelValidationMetrics {
private Integer epoch;
private Float aAcc;
private Float mFscore;
private Float mPrecision;
private Float mRecall;
private Float mIou;
private Float mAcc;
private Float changedFscore;
private Float changedPrecision;
private Float changedRecall;
private Float unchangedFscore;
private Float unchangedPrecision;
private Float unchangedRecall;
}
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class ModelTestMetrics {
private String model;
private Long tp;
private Long fp;
private Long fn;
private Float precision;
private Float recall;
private Float f1Score;
private Float accuracy;
private Float iou;
private Long detectionCount;
private Long gtCount;
}
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class ModelBestEpoch {
private Integer epoch;
private Double loss;
private Float f1Score;
private Float precision;
private Float recall;
private Float iou;
private Float accuracy;
}
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class ModelFileInfo {
private Boolean fileExistsYn;
private String fileName;
}
}

View File

@@ -0,0 +1,330 @@
package com.kamco.cd.training.model.dto;
import com.kamco.cd.training.common.dto.HyperParam;
import com.kamco.cd.training.common.enums.TrainStatusType;
import com.kamco.cd.training.common.enums.TrainType;
import com.kamco.cd.training.common.utils.interfaces.JsonFormatDttm;
import io.swagger.v3.oas.annotations.media.Schema;
import jakarta.validation.constraints.NotNull;
import java.time.Duration;
import java.time.ZonedDateTime;
import java.util.List;
import java.util.UUID;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import org.springframework.data.domain.PageRequest;
import org.springframework.data.domain.Pageable;
public class ModelTrainMngDto {
@Schema(name = "모델학습관리 목록", description = "모델학습관리 목록")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class Basic {
private Long id;
private UUID uuid;
private String modelVer;
@JsonFormatDttm private ZonedDateTime startDttm;
@JsonFormatDttm private ZonedDateTime step1StrtDttm;
@JsonFormatDttm private ZonedDateTime step1EndDttm;
@JsonFormatDttm private ZonedDateTime step2StrtDttm;
@JsonFormatDttm private ZonedDateTime step2EndDttm;
private String step1Status;
private String step2Status;
private String statusCd;
private String trainType;
private String modelNo;
private Long currentAttemptId;
private String requestPath;
private String packingState;
private ZonedDateTime packingStrtDttm;
private ZonedDateTime packingEndDttm;
private Long beforeModelId;
public String getStatusName() {
if (this.statusCd == null || this.statusCd.isBlank()) return null;
try {
return TrainStatusType.valueOf(this.statusCd).getText(); // 또는 getName()
} catch (IllegalArgumentException e) {
return this.statusCd; // 매핑 못하면 코드 그대로 반환(원하면 null 처리)
}
}
public String getStep1StatusName() {
if (this.step1Status == null || this.step1Status.isBlank()) return null;
try {
return TrainStatusType.valueOf(this.step1Status).getText(); // 또는 getName()
} catch (IllegalArgumentException e) {
return this.step1Status; // 매핑 못하면 코드 그대로 반환(원하면 null 처리)
}
}
public String getStep2StatusName() {
if (this.step2Status == null || this.step2Status.isBlank()) return null;
try {
return TrainStatusType.valueOf(this.step2Status).getText(); // 또는 getName()
} catch (IllegalArgumentException e) {
return this.step2Status; // 매핑 못하면 코드 그대로 반환(원하면 null 처리)
}
}
public String getTrainTypeName() {
if (this.trainType == null || this.trainType.isBlank()) return null;
try {
return TrainType.valueOf(this.trainType).getText(); // 또는 getName()
} catch (IllegalArgumentException e) {
return this.trainType; // 매핑 못하면 코드 그대로 반환(원하면 null 처리)
}
}
private String formatDuration(ZonedDateTime start, ZonedDateTime end) {
if (start == null || end == null) {
return null;
}
long totalSeconds = Math.abs(Duration.between(start, end).getSeconds());
long hours = totalSeconds / 3600;
long minutes = (totalSeconds % 3600) / 60;
long seconds = totalSeconds % 60;
return String.format("%d시간 %d분 %d초", hours, minutes, seconds);
}
public String getStep1Duration() {
return formatDuration(this.step1StrtDttm, this.step1EndDttm);
}
public String getStep2Duration() {
return formatDuration(this.step2StrtDttm, this.step2EndDttm);
}
public String getPackingDuration() {
return formatDuration(this.packingStrtDttm, this.packingEndDttm);
}
}
@Schema(name = "searchReq", description = "모델학습 관리 목록조회 파라미터")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class SearchReq {
private String status;
private String modelNo;
// 페이징 파라미터
private int page = 0;
private int size = 20;
public Pageable toPageable() {
return PageRequest.of(page, size);
}
}
@Schema(name = "addReq", description = "모델학습 관리 등록 파라미터")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class AddReq {
@NotNull
@Schema(description = "모델 종류 G1, G2, G3", example = "G1")
private String modelNo;
@NotNull
@Schema(description = "모델학습 실행 여부", example = "false")
private Boolean isStart;
@NotNull
@Schema(description = "학습타입 GENERAL(일반), TRANSFER(전이)", example = "GENERAL")
private String trainType;
@Schema(description = "전이학습일때 선택한 모델 id")
private Long beforeModelId;
@NotNull
@Schema(
description = "하이퍼 파라미터 선택 타입 OPTIMIZED(최적화 파라미터),EXISTING(기존 파라미터),NEW(신규 파라미터)",
example = "EXISTING")
private String hyperParamType;
@Schema(description = "하이퍼파라미터 uuid", example = "57fc9170-64c1-4128-aa7b-0657f08d6d10")
private UUID hyperUuid;
HyperParam hyperParam;
TrainingDataset trainingDataset;
ModelConfig modelConfig;
}
@Schema(name = "addReq", description = "모델학습 관리 등록 파라미터")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class UpdateReq {
private String requestPath;
private String responsePath;
}
@Getter
@Setter
public static class TrainingDataset {
Summary summary;
List<Long> datasetList;
}
@Getter
@Setter
public static class Summary {
@Schema(description = "건물", example = "0")
private Long buildingCnt;
@Schema(description = "컨테이너", example = "0")
private Long containerCnt;
@Schema(description = "폐기물", example = "0")
private Long wasteCnt;
@Schema(
description = "도로, 비닐하우스, 밭, 과수원, 초지, 숲, 물, 모재/자갈, 토분(무덤), 일반토지, 태양광, 기타",
example = "0")
private Long LandCoverCnt;
}
@Getter
@Setter
public static class ModelConfig {
@Schema(description = "에폭 횟수", example = "0")
private Integer epochCnt;
@Schema(description = "학습데이터셋 비율 Training", example = "0")
private Float trainingCnt;
@Schema(description = "학습데이터셋 비율 Validation", example = "0")
private Float validationCnt;
@Schema(description = "학습데이터셋 비율 Test", example = "0")
private Float testCnt;
@Schema(description = "메모", example = "메모 입니다.")
private String memo;
}
@Schema(name = "모델학습관리 목록", description = "모델학습관리 목록")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class ListDto {
private Long id;
private UUID uuid;
private String modelVer;
@JsonFormatDttm private ZonedDateTime startDttm;
@JsonFormatDttm private ZonedDateTime step1StrtDttm;
@JsonFormatDttm private ZonedDateTime step1EndDttm;
@JsonFormatDttm private ZonedDateTime step2StrtDttm;
@JsonFormatDttm private ZonedDateTime step2EndDttm;
private String step1Status;
private String step2Status;
private String statusCd;
private String trainType;
private String modelNo;
private Long currentAttemptId;
private String requestPath;
private String packingState;
private ZonedDateTime packingStrtDttm;
private ZonedDateTime packingEndDttm;
private String memo;
private String userNm;
private UUID beforeUuid;
public String getStatusName() {
if (this.statusCd == null || this.statusCd.isBlank()) return null;
try {
return TrainStatusType.valueOf(this.statusCd).getText(); // 또는 getName()
} catch (IllegalArgumentException e) {
return this.statusCd; // 매핑 못하면 코드 그대로 반환(원하면 null 처리)
}
}
public String getStep1StatusName() {
if (this.step1Status == null || this.step1Status.isBlank()) return null;
try {
return TrainStatusType.valueOf(this.step1Status).getText(); // 또는 getName()
} catch (IllegalArgumentException e) {
return this.step1Status; // 매핑 못하면 코드 그대로 반환(원하면 null 처리)
}
}
public String getStep2StatusName() {
if (this.step2Status == null || this.step2Status.isBlank()) return null;
try {
return TrainStatusType.valueOf(this.step2Status).getText(); // 또는 getName()
} catch (IllegalArgumentException e) {
return this.step2Status; // 매핑 못하면 코드 그대로 반환(원하면 null 처리)
}
}
public String getTrainTypeName() {
if (this.trainType == null || this.trainType.isBlank()) return null;
try {
return TrainType.valueOf(this.trainType).getText(); // 또는 getName()
} catch (IllegalArgumentException e) {
return this.trainType; // 매핑 못하면 코드 그대로 반환(원하면 null 처리)
}
}
private String formatDuration(ZonedDateTime start, ZonedDateTime end) {
if (start == null || end == null) {
return null;
}
long totalSeconds = Math.abs(Duration.between(start, end).getSeconds());
long hours = totalSeconds / 3600;
long minutes = (totalSeconds % 3600) / 60;
long seconds = totalSeconds % 60;
return String.format("%d시간 %d분 %d초", hours, minutes, seconds);
}
public String getStep1Duration() {
return formatDuration(this.step1StrtDttm, this.step1EndDttm);
}
public String getStep2Duration() {
return formatDuration(this.step2StrtDttm, this.step2EndDttm);
}
public String getPackingDuration() {
return formatDuration(this.packingStrtDttm, this.packingEndDttm);
}
}
@Getter
@Builder
@AllArgsConstructor
public static class ModelProgressStepDto {
private int step;
private String status;
@JsonFormatDttm private ZonedDateTime startTime;
@JsonFormatDttm private ZonedDateTime endTime;
private boolean isError;
}
}

View File

@@ -1,61 +0,0 @@
package com.kamco.cd.training.model.dto;
import com.kamco.cd.training.common.utils.interfaces.JsonFormatDttm;
import io.swagger.v3.oas.annotations.media.Schema;
import java.time.ZonedDateTime;
import lombok.Getter;
public class ModelVerDto {
@Schema(name = "modelVer Basic", description = "모델버전 엔티티 기본 정보")
@Getter
public static class Basic {
private final Long id;
private final Long modelUid;
private final String modelCate;
private final String modelVer;
private final String usedState;
private final String modelState;
private final Double qualityProb;
private final String deployState;
private final String modelPath;
@JsonFormatDttm private final ZonedDateTime createdDttm;
private final Long createdUid;
@JsonFormatDttm private final ZonedDateTime updatedDttm;
private final Long updatedUid;
public Basic(
Long id,
Long modelUid,
String modelCate,
String modelVer,
String usedState,
String modelState,
Double qualityProb,
String deployState,
String modelPath,
ZonedDateTime createdDttm,
Long createdUid,
ZonedDateTime updatedDttm,
Long updatedUid) {
this.id = id;
this.modelUid = modelUid;
this.modelCate = modelCate;
this.modelVer = modelVer;
this.usedState = usedState;
this.modelState = modelState;
this.qualityProb = qualityProb;
this.deployState = deployState;
this.modelPath = modelPath;
this.createdDttm = createdDttm;
this.createdUid = createdUid;
this.updatedDttm = updatedDttm;
this.updatedUid = updatedUid;
}
}
}

View File

@@ -1,50 +0,0 @@
package com.kamco.cd.training.model.service;
import com.kamco.cd.training.model.dto.ModelMngDto;
import com.kamco.cd.training.model.dto.ModelMngDto.Basic;
import com.kamco.cd.training.model.dto.ModelMngDto.SearchReq;
import com.kamco.cd.training.postgres.core.ModelMngCoreService;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.data.domain.Page;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
@Service
@RequiredArgsConstructor
@Transactional(readOnly = true)
@Slf4j
public class ModelMngService {
private final ModelMngCoreService modelMngCoreService;
/**
* 모델 목록 조회
*
* @param searchReq 검색 조건
* @return 페이징 처리된 모델 목록
*/
public Page<Basic> findByModels(SearchReq searchReq) {
return modelMngCoreService.findByModels(searchReq);
}
/**
* 모델 상세 조회
*
* @param modelUid 모델 UID
* @return 모델 상세 정보
*/
public ModelMngDto.Detail getModelDetail(Long modelUid) {
return modelMngCoreService.getModelDetail(modelUid);
}
/**
* 모델 상세 조회 (UUID 기반)
*
* @param uuid 모델 UUID
* @return 모델 상세 정보
*/
public ModelMngDto.Detail getModelDetailByUuid(String uuid) {
return modelMngCoreService.getModelDetailByUuid(uuid);
}
}

View File

@@ -0,0 +1,155 @@
package com.kamco.cd.training.model.service;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetReq;
import com.kamco.cd.training.dataset.dto.DatasetDto.SelectTransferDataSet;
import com.kamco.cd.training.model.dto.ModelConfigDto;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.DetailSummary;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.HyperSummary;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.MappingDataset;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelBestEpoch;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelFileInfo;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelTestMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelTrainMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelValidationMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.TransferDetailDto;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.TransferHyperSummary;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.Basic;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.ModelProgressStepDto;
import com.kamco.cd.training.postgres.core.ModelTrainDetailCoreService;
import com.kamco.cd.training.postgres.core.ModelTrainMngCoreService;
import java.util.ArrayList;
import java.util.List;
import java.util.UUID;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
@Service
@RequiredArgsConstructor
@Transactional(readOnly = true)
@Slf4j
public class ModelTrainDetailService {
private final ModelTrainDetailCoreService modelTrainDetailCoreService;
private final ModelTrainMngCoreService mngCoreService;
/**
* 모델 상세정보 요약
*
* @param uuid
* @return
*/
public DetailSummary getModelDetailSummary(UUID uuid) {
return modelTrainDetailCoreService.getModelDetailSummary(uuid);
}
/**
* 모델 하이퍼파라미터 요약
*
* @param uuid
* @return
*/
public HyperSummary getByModelHyperParamSummary(UUID uuid) {
return modelTrainDetailCoreService.getByModelHyperParamSummary(uuid);
}
public List<MappingDataset> getByModelMappingDataset(UUID uuid) {
return modelTrainDetailCoreService.getByModelMappingDataset(uuid);
}
public Basic findByModelByUUID(UUID uuid) {
return modelTrainDetailCoreService.findByModelByUUID(uuid);
}
/**
* 전이학습 모델선택 정보
*
* @param uuid
* @return
*/
public TransferDetailDto getTransferDetail(UUID uuid) {
Basic modelInfo = modelTrainDetailCoreService.findByModelByUUID(uuid);
// config 정보 조회
ModelConfigDto.TransferBasic configInfo = mngCoreService.findModelTransferConfigByModelId(uuid);
// 하이파라미터 정보 조회
TransferHyperSummary hyperSummary = modelTrainDetailCoreService.getTransferHyperSummary(uuid);
List<SelectTransferDataSet> dataSets = new ArrayList<>();
DatasetReq datasetReq = new DatasetReq();
List<Long> datasetIds = new ArrayList<>();
List<MappingDataset> mappingDatasets =
modelTrainDetailCoreService.getByModelMappingDataset(uuid);
for (MappingDataset mappingDataset : mappingDatasets) {
datasetIds.add(mappingDataset.getDatasetId());
}
datasetReq.setIds(datasetIds);
datasetReq.setModelNo(modelInfo.getModelNo());
if (modelInfo.getModelNo().equals(ModelType.G1.getId())) {
dataSets = mngCoreService.getDatasetTransferSelectG1List(modelInfo.getId());
} else {
dataSets =
mngCoreService.getDatasetTransferSelectG2G3List(
modelInfo.getId(), modelInfo.getModelNo());
}
// DatasetReq beforeDatasetReq = new DatasetReq();
// List<Long> beforeDatasetIds = new ArrayList<>();
// List<SelectDataSet> beforeDataSets = new ArrayList<>();
//
// Long beforeModelId = modelInfo.getBeforeModelId();
// if (beforeModelId != null) {
// Basic beforeInfo = modelTrainDetailCoreService.findByModelBeforeId(beforeModelId);
// List<MappingDataset> beforeDatasets =
// modelTrainDetailCoreService.getByModelMappingDataset(beforeInfo.getUuid());
//
// for (MappingDataset before : beforeDatasets) {
// beforeDatasetIds.add(before.getDatasetId());
// }
// beforeDatasetReq.setIds(beforeDatasetIds);
// beforeDatasetReq.setModelNo(modelInfo.getModelNo());
//
// if (beforeInfo.getModelNo().equals(ModelType.G1.getId())) {
// beforeDataSets = mngCoreService.getDatasetSelectG1List(beforeDatasetReq);
// } else {
// beforeDataSets = mngCoreService.getDatasetSelectG2G3List(beforeDatasetReq);
// }
// }
TransferDetailDto transferDetailDto = new TransferDetailDto();
transferDetailDto.setEtcConfig(configInfo);
transferDetailDto.setModelTrainHyper(hyperSummary);
transferDetailDto.setModelTrainDataset(dataSets);
return transferDetailDto;
}
public List<ModelTrainMetrics> getModelTrainMetricResult(UUID uuid) {
return modelTrainDetailCoreService.getModelTrainMetricResult(uuid);
}
public List<ModelValidationMetrics> getModelValidationMetricResult(UUID uuid) {
return modelTrainDetailCoreService.getModelValidationMetricResult(uuid);
}
public List<ModelTestMetrics> getModelTestMetricResult(UUID uuid) {
return modelTrainDetailCoreService.getModelTestMetricResult(uuid);
}
public ModelBestEpoch getModelTrainBestEpoch(UUID uuid) {
return modelTrainDetailCoreService.getModelTrainBestEpoch(uuid);
}
public ModelFileInfo getModelTrainFileInfo(UUID uuid) {
return modelTrainDetailCoreService.getModelTrainFileInfo(uuid);
}
public List<ModelProgressStepDto> findModelTrainProgressInfo(UUID uuid) {
return modelTrainDetailCoreService.findModelTrainProgressInfo(uuid);
}
}

View File

@@ -0,0 +1,129 @@
package com.kamco.cd.training.model.service;
import com.kamco.cd.training.common.dto.HyperParam;
import com.kamco.cd.training.common.enums.HyperParamSelectType;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.common.enums.TrainType;
import com.kamco.cd.training.common.exception.CustomApiException;
import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetReq;
import com.kamco.cd.training.dataset.dto.DatasetDto.SelectDataSet;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto;
import com.kamco.cd.training.model.dto.ModelConfigDto;
import com.kamco.cd.training.model.dto.ModelTrainMngDto;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.SearchReq;
import com.kamco.cd.training.postgres.core.HyperParamCoreService;
import com.kamco.cd.training.postgres.core.ModelTrainMngCoreService;
import com.kamco.cd.training.train.service.TrainJobService;
import java.util.List;
import java.util.UUID;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.data.domain.Page;
import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
@Service
@RequiredArgsConstructor
@Transactional(readOnly = true)
@Slf4j
public class ModelTrainMngService {
private final ModelTrainMngCoreService modelTrainMngCoreService;
private final HyperParamCoreService hyperParamCoreService;
private final TrainJobService trainJobService;
/**
* 모델학습 조회
*
* @param searchReq 검색 조건
* @return 페이징 처리된 모델 목록
*/
public Page<ModelTrainMngDto.ListDto> getModelList(SearchReq searchReq) {
return modelTrainMngCoreService.findByModelList(searchReq);
}
/**
* 모델학습 삭제
*
* @param uuid
*/
@Transactional
public void deleteModelTrain(UUID uuid) {
modelTrainMngCoreService.deleteModel(uuid);
}
/**
* 모델학습 등록
*
* @param req
* @return
*/
@Transactional
public UUID createModelTrain(ModelTrainMngDto.AddReq req) {
HyperParam hyperParam = req.getHyperParam();
HyperParamDto.Basic hyper = new HyperParamDto.Basic();
// 전이 학습은 모델 선택 필수
if (TrainType.TRANSFER.getId().equals(req.getTrainType())) {
if (req.getBeforeModelId() == null) {
throw new CustomApiException("BAD_REQUEST", HttpStatus.BAD_REQUEST, "모델을 선택해 주세요.");
}
}
// 하이파라미터 신규저장
if (HyperParamSelectType.NEW.getId().equals(req.getHyperParamType())) {
// 하이퍼파라미터 등록
hyper = hyperParamCoreService.createHyperParam(hyperParam);
req.setHyperUuid(hyper.getUuid());
}
// 모델학습 테이블 저장
ModelTrainMngDto.Basic modelDto = modelTrainMngCoreService.saveModel(req);
Long modelId = modelDto.getId();
UUID modelUuid = modelDto.getUuid();
// 모델학습 데이터셋 저장
modelTrainMngCoreService.saveModelDataset(modelId, req);
// 모델 데이터셋 mapping 저장
modelTrainMngCoreService.saveModelDatasetMap(
modelId, req.getTrainingDataset().getDatasetList());
// 모델 config 저장
modelTrainMngCoreService.saveModelConfig(modelId, req.getModelConfig());
// 데이터셋 임시파일 생성
trainJobService.createTmpFile(modelUuid);
return modelUuid;
}
/**
* 모델학습 모델학습 uuid config정보 조회
*
* @param uuid 모델학습 uuid
* @return
*/
public ModelConfigDto.Basic getModelConfigByModelId(UUID uuid) {
return modelTrainMngCoreService.findModelConfigByModelId(uuid);
}
/**
* 모델별 데이터셋 목록 조회
*
* @param req
* @return
*/
public List<SelectDataSet> getDatasetSelectList(DatasetReq req) {
if (req.getModelNo().equals(ModelType.G1.getId())) {
return modelTrainMngCoreService.getDatasetSelectG1List(req);
} else {
return modelTrainMngCoreService.getDatasetSelectG2G3List(req);
}
}
public Long findModelStep1InProgressCnt() {
return modelTrainMngCoreService.findModelStep1InProgressCnt();
}
}

View File

@@ -1,353 +0,0 @@
package com.kamco.cd.training.model.service;
import com.kamco.cd.training.common.exception.BadRequestException;
import com.kamco.cd.training.common.exception.NotFoundException;
import com.kamco.cd.training.model.dto.ModelMngDto;
import com.kamco.cd.training.postgres.core.DatasetCoreService;
import com.kamco.cd.training.postgres.core.HyperParamCoreService;
import com.kamco.cd.training.postgres.core.ModelMngCoreService;
import com.kamco.cd.training.postgres.core.SystemMetricsCoreService;
import com.kamco.cd.training.postgres.entity.ModelTrainMasterEntity;
import java.util.List;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
@Service
@RequiredArgsConstructor
@Transactional(readOnly = true)
@Slf4j
public class ModelTrainService {
private final ModelMngCoreService modelMngCoreService;
private final HyperParamCoreService hyperParamCoreService;
private final DatasetCoreService datasetCoreService;
private final SystemMetricsCoreService systemMetricsCoreService;
/**
* 학습 모델 목록 조회
*
* @return 학습 모델 목록
*/
public List<ModelMngDto.TrainListRes> getTrainModelList() {
return modelMngCoreService.findAllTrainModels();
}
/**
* 학습 설정 통합 조회
*
* @return 학습 설정 폼 데이터
*/
public ModelMngDto.FormConfigRes getFormConfig() {
// 1. 현재 실행 중인 모델 확인
String runningModelUuid = modelMngCoreService.findRunningModelUuid();
boolean isTrainAvailable = (runningModelUuid == null);
// 2. 저장공간 체크 (10GB 미만 시 학습 불가)
if (isTrainAvailable) {
isTrainAvailable = systemMetricsCoreService.isStorageAvailableForTraining();
long availableMB = systemMetricsCoreService.getAvailableStorageMB();
log.info("저장공간 체크 완료: {}MB 사용 가능, 학습 가능 여부: {}", availableMB, isTrainAvailable);
}
// 3. 하이퍼파라미터 목록
List<ModelMngDto.HyperParamInfo> hyperParams =
null; // hyperParamCoreService.findAllActiveHyperParams();
// 4. 데이터셋 목록
List<ModelMngDto.DatasetInfo> datasets = datasetCoreService.findAllActiveDatasetsForTraining();
return ModelMngDto.FormConfigRes.builder()
.isTrainAvailable(isTrainAvailable)
.hyperParams(hyperParams)
.datasets(datasets)
.runningModelUuid(runningModelUuid)
.build();
}
/**
* 학습 시작
*
* @param trainReq 학습 시작 요청
* @return 학습 시작 응답
*/
@Transactional
public ModelMngDto.TrainStartRes startTraining(ModelMngDto.TrainStartReq trainReq) {
// 1. 동시 실행 방지 검증
String runningModelUuid = modelMngCoreService.findRunningModelUuid();
if (runningModelUuid != null) {
throw new BadRequestException(
"이미 실행 중인 학습이 있습니다. 학습은 한 번에 한 개만 실행할 수 있습니다. (실행 중인 모델: " + runningModelUuid + ")");
}
// 2. 저장공간 체크 (10GB 미만 시 학습 불가)
if (!systemMetricsCoreService.isStorageAvailableForTraining()) {
long availableMB = systemMetricsCoreService.getAvailableStorageMB();
long requiredMB = 10 * 1024; // 10GB
throw new BadRequestException(
String.format(
"저장공간이 부족하여 학습을 시작할 수 없습니다. (필요: %dMB, 사용 가능: %dMB)", requiredMB, availableMB));
}
// 3. 데이터셋 상태 검증 (COMPLETED 상태만 학습 가능)
validateDatasetStatus(trainReq.getDatasetIds());
// 4. 데이터 분할 비율 검증 (예: "7:2:1" 형식)
if (trainReq.getDatasetRatio() != null && !trainReq.getDatasetRatio().isEmpty()) {
validateDatasetRatio(trainReq.getDatasetRatio());
}
// 5. 학습 마스터 생성
ModelTrainMasterEntity entity = modelMngCoreService.createTrainMaster(trainReq);
// 5. 데이터셋 매핑 생성
modelMngCoreService.createDatasetMappings(entity.getId(), trainReq.getDatasetIds());
// 6. 실제 UUID 사용
String uuid = entity.getUuid().toString();
log.info(
"학습 시작: uuid={}, hyperVer={}, epoch={}, datasets={}",
uuid,
trainReq.getHyperVer(),
trainReq.getEpoch(),
trainReq.getDatasetIds());
// TODO: 비동기 GPU 학습 프로세스 트리거 로직 추가
return ModelMngDto.TrainStartRes.builder().uuid(uuid).status(entity.getStatusCd()).build();
}
/**
* 데이터셋 상태 검증
*
* @param datasetIds 데이터셋 ID 목록
*/
private void validateDatasetStatus(List<Long> datasetIds) {
for (Long datasetId : datasetIds) {
try {
var dataset = datasetCoreService.getOneById(datasetId);
// COMPLETED 상태가 아닌 데이터셋이 포함되어 있으면 예외 발생
if (dataset.getStatus() == null || !"COMPLETED".equals(dataset.getStatus())) {
throw new BadRequestException(
String.format(
"학습에 사용할 수 없는 데이터셋입니다. (ID: %d, 상태: %s). COMPLETED 상태의 데이터셋만 선택 가능합니다.",
datasetId, dataset.getStatus() != null ? dataset.getStatus() : "NULL"));
}
log.debug("데이터셋 상태 검증 통과: ID={}, Status={}", datasetId, dataset.getStatus());
} catch (NotFoundException e) {
throw new BadRequestException("존재하지 않는 데이터셋입니다. ID: " + datasetId);
}
}
log.info("모든 데이터셋 상태 검증 완료: {} 개", datasetIds.size());
}
/**
* 데이터 분할 비율 검증
*
* @param datasetRatio 데이터셋 비율 (예: "7:2:1")
*/
private void validateDatasetRatio(String datasetRatio) {
try {
String[] parts = datasetRatio.split(":");
if (parts.length != 3) {
throw new BadRequestException("데이터 분할 비율은 'Training:Validation:Test' 형식이어야 합니다 (예: 7:2:1)");
}
int train = Integer.parseInt(parts[0].trim());
int validation = Integer.parseInt(parts[1].trim());
int test = Integer.parseInt(parts[2].trim());
int sum = train + validation + test;
if (sum != 10) {
throw new BadRequestException(
String.format("데이터 분할 비율의 합계는 10이어야 합니다. (현재 합계: %d, 입력값: %s)", sum, datasetRatio));
}
if (train <= 0 || validation < 0 || test < 0) {
throw new BadRequestException("데이터 분할 비율은 모두 0 이상이어야 합니다 (Training은 1 이상)");
}
log.info(
"데이터 분할 비율 검증 완료: Training={}0%, Validation={}0%, Test={}0%", train, validation, test);
} catch (NumberFormatException e) {
throw new BadRequestException("데이터 분할 비율은 숫자로만 구성되어야 합니다: " + datasetRatio);
}
}
/**
* 학습 모델 삭제
*
* @param uuid 모델 UUID
*/
@Transactional
public void deleteTrainModel(String uuid) {
modelMngCoreService.deleteByUuid(uuid);
log.info("학습 모델 삭제 완료: uuid={}", uuid);
}
// ==================== Resume Training (학습 재시작) ====================
/**
* 학습 재시작 정보 조회
*
* @param uuid 모델 UUID
* @return 재시작 정보
*/
public ModelMngDto.ResumeInfo getResumeInfo(String uuid) {
ModelTrainMasterEntity entity = modelMngCoreService.findByUuid(uuid);
return ModelMngDto.ResumeInfo.builder()
.canResume(entity.getCanResume() != null && entity.getCanResume())
.lastEpoch(entity.getLastCheckpointEpoch())
.totalEpoch(entity.getEpochCnt())
.checkpointPath(entity.getCheckpointPath())
// .failedAt(
// entity.getStopDttm() != null
// ? entity.getStopDttm().atZone(java.time.ZoneId.systemDefault())
// : null)
.build();
}
/**
* 학습 재시작
*
* @param uuid 모델 UUID
* @param resumeReq 재시작 요청
* @return 재시작 응답
*/
@Transactional
public ModelMngDto.ResumeResponse resumeTraining(
String uuid, ModelMngDto.ResumeRequest resumeReq) {
ModelTrainMasterEntity entity = modelMngCoreService.findByUuid(uuid);
// 재시작 가능 여부 검증
if (entity.getCanResume() == null || !entity.getCanResume()) {
throw new IllegalStateException("학습 재시작이 불가능한 모델입니다: " + uuid);
}
if (entity.getLastCheckpointEpoch() == null) {
throw new IllegalStateException("Checkpoint가 존재하지 않습니다: " + uuid);
}
// 상태 업데이트
entity.setStatusCd("RUNNING");
entity.setProgressRate(0);
// 총 Epoch 수 변경 (선택사항)
if (resumeReq.getNewTotalEpoch() != null) {
entity.setEpochCnt(resumeReq.getNewTotalEpoch());
}
log.info(
"학습 재시작: uuid={}, resumeFromEpoch={}, totalEpoch={}",
uuid,
resumeReq.getResumeFromEpoch(),
entity.getEpochCnt());
// TODO: 비동기 GPU 학습 재시작 프로세스 트리거 로직 추가
// - Checkpoint 파일 로드
// - 지정된 Epoch부터 학습 재개
return ModelMngDto.ResumeResponse.builder()
.uuid(uuid)
.status(entity.getStatusCd())
.resumedFromEpoch(resumeReq.getResumeFromEpoch())
.build();
}
// ==================== Best Epoch Setting (Best Epoch 설정) ====================
/**
* Best Epoch 설정
*
* @param uuid 모델 UUID
* @param bestEpochReq Best Epoch 요청
* @return Best Epoch 응답
*/
@Transactional
public ModelMngDto.BestEpochResponse setBestEpoch(
String uuid, ModelMngDto.BestEpochRequest bestEpochReq) {
ModelTrainMasterEntity entity = modelMngCoreService.findByUuid(uuid);
// 1차 학습 완료 상태 검증
if (!"STEP1_COMPLETED".equals(entity.getStatusCd())
&& !"STEP1".equals(entity.getProcessStep())) {
log.warn(
"Best Epoch 설정 시도: 현재 상태={}, processStep={}",
entity.getStatusCd(),
entity.getProcessStep());
}
Integer previousBestEpoch = entity.getConfirmedBestEpoch();
// 사용자가 확정한 Best Epoch 설정
entity.setConfirmedBestEpoch(bestEpochReq.getBestEpoch());
// 2차 학습(Test) 단계로 상태 전이
entity.setProcessStep("STEP2");
entity.setStatusCd("STEP2_RUNNING");
entity.setProgressRate(0);
entity.setUpdatedDttm(java.time.ZonedDateTime.now());
log.info(
"Best Epoch 설정 및 2차 학습 시작: uuid={}, newBestEpoch={}, previousBestEpoch={}, reason={}, newStatus={}",
uuid,
bestEpochReq.getBestEpoch(),
previousBestEpoch,
bestEpochReq.getReason(),
entity.getStatusCd());
// TODO: 비동기 GPU 2차 학습(Test) 프로세스 트리거 로직 추가
// - Best Epoch 모델 로드
// - Test 데이터셋으로 성능 평가 실행
// - 완료 시 STEP2_COMPLETED 상태로 전환
return ModelMngDto.BestEpochResponse.builder()
.uuid(uuid)
.bestEpoch(entity.getBestEpoch()) // 자동 선택된 값
.confirmedBestEpoch(entity.getConfirmedBestEpoch()) // 사용자 확정 값
.previousBestEpoch(previousBestEpoch)
.build();
}
/**
* Epoch별 성능 지표 조회
*
* @param uuid 모델 UUID
* @return Epoch별 성능 지표 목록
*/
public List<ModelMngDto.EpochMetric> getEpochMetrics(String uuid) {
ModelTrainMasterEntity entity = modelMngCoreService.findByUuid(uuid);
// TODO: 실제 학습 로그 파일이나 DB에서 Epoch별 성능 지표 조회
// 현재는 샘플 데이터 반환
List<ModelMngDto.EpochMetric> metrics = new java.util.ArrayList<>();
if (entity.getEpochCnt() != null && entity.getBestEpoch() != null) {
// 샘플 데이터 생성 (실제로는 학습 로그 파일 파싱 또는 별도 테이블 조회)
for (int i = 1; i <= Math.min(entity.getEpochCnt(), 10); i++) {
int epoch = entity.getBestEpoch() - 5 + i;
if (epoch <= 0 || epoch > entity.getEpochCnt()) {
continue;
}
metrics.add(
ModelMngDto.EpochMetric.builder()
.epoch(epoch)
.mIoU(0.80 + (Math.random() * 0.15)) // 샘플 데이터
.mFscore(0.85 + (Math.random() * 0.10)) // 샘플 데이터
.loss(0.3 - (Math.random() * 0.15)) // 샘플 데이터
.isBest(entity.getBestEpoch() != null && epoch == entity.getBestEpoch())
.build());
}
}
log.info("Epoch별 성능 지표 조회: uuid={}, metricsCount={}", uuid, metrics.size());
return metrics;
}
}

View File

@@ -2,6 +2,7 @@ package com.kamco.cd.training.postgres.core;
import com.kamco.cd.training.common.service.BaseCoreService;
import com.kamco.cd.training.log.dto.AuditLogDto;
import com.kamco.cd.training.log.dto.AuditLogDto.DownloadReq;
import com.kamco.cd.training.postgres.repository.log.AuditLogRepository;
import java.time.LocalDate;
import lombok.RequiredArgsConstructor;
@@ -45,6 +46,11 @@ public class AuditLogCoreService
return auditLogRepository.findLogByAccount(searchRange, searchValue);
}
public Page<AuditLogDto.DownloadRes> findLogByAccount(
AuditLogDto.searchReq searchReq, DownloadReq downloadReq) {
return auditLogRepository.findDownloadLog(searchReq, downloadReq);
}
public Page<AuditLogDto.DailyDetail> getLogByDailyResult(
AuditLogDto.searchReq searchRange, LocalDate logDate) {
return auditLogRepository.findLogByDailyResult(searchRange, logDate);

View File

@@ -1,17 +1,23 @@
package com.kamco.cd.training.postgres.core;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.kamco.cd.training.common.enums.LearnDataRegister;
import com.kamco.cd.training.common.enums.LearnDataType;
import com.kamco.cd.training.common.exception.NotFoundException;
import com.kamco.cd.training.common.service.BaseCoreService;
import com.kamco.cd.training.dataset.dto.DatasetDto;
import com.kamco.cd.training.model.dto.ModelMngDto;
import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetMngRegDto;
import com.kamco.cd.training.dataset.dto.DatasetObjDto.Basic;
import com.kamco.cd.training.dataset.dto.DatasetObjDto.DatasetClass;
import com.kamco.cd.training.dataset.dto.DatasetObjDto.DatasetObjRegDto;
import com.kamco.cd.training.dataset.dto.DatasetObjDto.SearchReq;
import com.kamco.cd.training.postgres.entity.DatasetEntity;
import com.kamco.cd.training.postgres.entity.DatasetObjEntity;
import com.kamco.cd.training.postgres.repository.dataset.DatasetObjRepository;
import com.kamco.cd.training.postgres.repository.dataset.DatasetRepository;
import jakarta.transaction.Transactional;
import java.time.ZonedDateTime;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.UUID;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
@@ -25,6 +31,7 @@ import org.springframework.stereotype.Service;
public class DatasetCoreService
implements BaseCoreService<DatasetDto.Basic, Long, DatasetDto.SearchReq> {
private final DatasetRepository datasetRepository;
private final DatasetObjRepository datasetObjRepository;
private final ObjectMapper objectMapper;
/**
@@ -99,25 +106,26 @@ public class DatasetCoreService
// 먼저 id1 필드를 임시값(0)으로 설정하여 저장
DatasetEntity entity = new DatasetEntity();
entity.setTitle(registerReq.getTitle());
entity.setYear(registerReq.getYear());
entity.setGroupTitle("PRODUCTION");
entity.setDataYear(registerReq.getYear());
// entity.setYear(registerReq.getYear());
entity.setDataType(LearnDataType.PRODUCTION.getId());
entity.setCompareYyyy(registerReq.getCompareYear());
entity.setTargetYyyy(registerReq.getTargetYyyy());
entity.setRoundNo(registerReq.getRoundNo() != null ? registerReq.getRoundNo() : 1L);
entity.setMemo(registerReq.getMemo());
entity.setStatus("READY");
entity.setStatus(LearnDataRegister.READY.getId());
entity.setDataType("CREATE");
entity.setTotalItems(0L);
entity.setTotalSize(0L);
entity.setItemCount(0L);
entity.setDeleted(false);
entity.setCreatedDttm(ZonedDateTime.now());
entity.setId1(0L); // 임시값
// entity.setId1(0L); // 임시값
DatasetEntity savedEntity = datasetRepository.save(entity);
// 저장 후 id1을 dataset_uid와 동일하게 업데이트
savedEntity.setId1(savedEntity.getId());
savedEntity = datasetRepository.save(savedEntity);
// savedEntity.setId1(savedEntity.getId());
// savedEntity = datasetRepository.save(savedEntity);
return savedEntity.toDto();
}
@@ -178,99 +186,68 @@ public class DatasetCoreService
summaryReq.getDatasetIds().size(), totalMapSheets, totalFileSize, averageMapSheets);
}
/**
* 활성 데이터셋 전체 조회 (학습 관리용)
*
* @return 데이터셋 정보 목록
*/
public List<ModelMngDto.DatasetInfo> findAllActiveDatasetsForTraining() {
List<DatasetEntity> entities = datasetRepository.findByDeletedOrderByCreatedDttmDesc(false);
public Page<Basic> searchDatasetObjectList(SearchReq searchReq) {
Page<DatasetObjEntity> entityPage = datasetObjRepository.searchDatasetObjectList(searchReq);
return entityPage.map(DatasetObjEntity::toDto);
}
return entities.stream()
.map(
entity -> {
// totalSize를 읽기 쉬운 형식으로 변환
String totalSizeStr = formatFileSize(entity.getTotalSize());
// classCounts JSON 파싱
Map<String, Integer> classCounts = entity.getClassCounts();
return ModelMngDto.DatasetInfo.builder()
.id(entity.getId())
.title(entity.getTitle())
.groupTitle(entity.getGroupTitle())
.totalItems(entity.getTotalItems())
.totalSize(totalSizeStr)
.classCounts(classCounts)
.memo(entity.getMemo())
.createdDttm(entity.getCreatedDttm())
.build();
})
.toList();
public UUID deleteDatasetObjByUuid(UUID uuid) {
DatasetObjEntity entity =
datasetObjRepository
.findByUuid(uuid)
.orElseThrow(() -> new NotFoundException("데이터셋 obj를 찾을 수 없습니다. ID: " + uuid));
entity.setDeleted(true);
datasetObjRepository.save(entity);
return entity.getUuid();
}
/**
* JSON 문자열을 Map으로 파싱
* 데이터셋 object class 조회
*
* @param jsonStr JSON 문자열
* @return 클래스별 카운트 맵
* @param uuid dataset uuid
* @param type compare, target
* @return
*/
private Map<String, Integer> parseClassCounts(String jsonStr) {
if (jsonStr == null || jsonStr.trim().isEmpty()) {
return new HashMap<>();
}
try {
return objectMapper.readValue(jsonStr, new TypeReference<Map<String, Integer>>() {});
} catch (Exception e) {
log.warn("클래스 통계 JSON 파싱 실패: {}", jsonStr, e);
return new HashMap<>();
}
public List<DatasetClass> findDatasetObjClassByUuid(UUID uuid, String type) {
return datasetObjRepository.findDatasetObjClassByUuid(uuid, type);
}
/**
* 데이터셋의 클래스 통계 계산 및 저장
*
* @param datasetId 데이터셋 ID
* @param classCounts 클래스별 카운트
*/
public void updateClassCounts(Long datasetId, Map<String, Integer> classCounts) {
public Long getDatasetMaxStage(int compareYyyy, int targetYyyy) {
return datasetRepository.getDatasetMaxStage(compareYyyy, targetYyyy);
}
@Transactional
public Long insertDatasetMngData(DatasetMngRegDto mngRegDto) {
return datasetRepository.insertDatasetMngData(mngRegDto);
}
@Transactional
public void insertDatasetObj(DatasetObjRegDto objRegDto) {
datasetObjRepository.insertDatasetObj(objRegDto);
}
public String getFilePathByUUIDPathType(UUID uuid, String pathType) {
return datasetObjRepository.getFilePathByUUIDPathType(uuid, pathType);
}
public void insertDatasetTestObj(DatasetObjRegDto objRegDto) {
datasetObjRepository.insertDatasetTestObj(objRegDto);
}
public void updateDatasetUploadStatus(Long datasetUid) {
DatasetEntity entity =
datasetRepository
.findById(datasetId)
.orElseThrow(() -> new NotFoundException("데이터셋을 찾을 수 없습니다. ID: " + datasetId));
.findById(datasetUid)
.orElseThrow(() -> new NotFoundException("데이터셋을 찾을 수 없습니다. ID: " + datasetUid));
try {
entity.setClassCounts(classCounts);
datasetRepository.save(entity);
log.info("데이터셋 클래스 통계 업데이트 완료: datasetId={}, classes={}", datasetId, classCounts.keySet());
} catch (Exception e) {
log.error("클래스 통계 JSON 변환 실패: datasetId={}", datasetId, e);
}
entity.setStatus(LearnDataRegister.COMPLETED.getId());
}
/**
* 파일 크기를 읽기 쉬운 형식으로 변환
*
* @param size 바이트 단위 크기
* @return 형식화된 문자열 (예: "1.5GB")
*/
private String formatFileSize(Long size) {
if (size == null || size == 0) {
return "0 GB";
}
public void insertDatasetValObj(DatasetObjRegDto objRegDto) {
datasetObjRepository.insertDatasetValObj(objRegDto);
}
double gb = size / (1024.0 * 1024.0 * 1024.0);
if (gb >= 1.0) {
return String.format("%.2f GB", gb);
}
double mb = size / (1024.0 * 1024.0);
if (mb >= 1.0) {
return String.format("%.2f MB", mb);
}
double kb = size / 1024.0;
return String.format("%.2f KB", kb);
public Long findDatasetByUidExistsCnt(String uid) {
return datasetRepository.findDatasetByUidExistsCnt(uid);
}
}

View File

@@ -1,8 +1,12 @@
package com.kamco.cd.training.postgres.core;
import com.kamco.cd.training.common.dto.HyperParam;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.common.exception.CustomApiException;
import com.kamco.cd.training.common.utils.UserUtil;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto.Basic;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto.SearchReq;
import com.kamco.cd.training.postgres.entity.ModelHyperParamEntity;
import com.kamco.cd.training.postgres.repository.hyperparam.HyperParamRepository;
import java.time.ZonedDateTime;
@@ -15,28 +19,31 @@ import org.springframework.stereotype.Service;
@Service
@RequiredArgsConstructor
public class HyperParamCoreService {
private final HyperParamRepository hyperParamRepository;
private final UserUtil userUtil;
/**
* 하이퍼파라미터 등록
*
* @param createReq 등록 요청
* @param createReq ModelTrainMngDto.HyperParamDto
* @return 등록된 버전명
*/
public String createHyperParam(HyperParamDto.HyperParamCreateReq createReq) {
String firstVersion = getFirstHyperParamVersion();
public Basic createHyperParam(HyperParam createReq) {
String firstVersion = getFirstHyperParamVersion(createReq.getModel());
ModelHyperParamEntity entity = new ModelHyperParamEntity();
entity.setHyperVer(firstVersion);
applyHyperParam(entity, createReq);
// user
entity.setCreatedUid(userUtil.getId());
ModelHyperParamEntity resultEntity = hyperParamRepository.save(entity);
return resultEntity.getHyperVer();
Basic basic = new Basic();
basic.setUuid(resultEntity.getUuid());
basic.setHyperVer(resultEntity.getHyperVer());
return basic;
}
/**
@@ -46,12 +53,15 @@ public class HyperParamCoreService {
* @param createReq 등록 요청
* @return ver
*/
public String updateHyperParam(UUID uuid, HyperParamDto.HyperParamCreateReq createReq) {
public String updateHyperParam(UUID uuid, HyperParam createReq) {
ModelHyperParamEntity entity =
hyperParamRepository
.findHyperParamByUuid(uuid)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
if (entity.getIsDefault()) {
throw new CustomApiException("UNPROCESSABLE_ENTITY_UPDATE", HttpStatus.UNPROCESSABLE_ENTITY);
}
applyHyperParam(entity, createReq);
// user
@@ -61,49 +71,6 @@ public class HyperParamCoreService {
return entity.getHyperVer();
}
private void applyHyperParam(
ModelHyperParamEntity entity, HyperParamDto.HyperParamCreateReq createReq) {
// Important
entity.setBackbone(createReq.getBackbone());
entity.setInputSize(createReq.getInputSize());
entity.setCropSize(createReq.getCropSize());
entity.setBatchSize(createReq.getBatchSize());
// Data
entity.setTrainNumWorkers(createReq.getTrainNumWorkers());
entity.setValNumWorkers(createReq.getValNumWorkers());
entity.setTestNumWorkers(createReq.getTestNumWorkers());
entity.setTrainShuffle(createReq.getTrainShuffle());
entity.setTrainPersistent(createReq.getTrainPersistent());
entity.setValPersistent(createReq.getValPersistent());
// Model Architecture
entity.setDropPathRate(createReq.getDropPathRate());
entity.setFrozenStages(createReq.getFrozenStages());
entity.setNeckPolicy(createReq.getNeckPolicy());
entity.setClassWeight(createReq.getClassWeight());
entity.setDecoderChannels(createReq.getDecoderChannels());
// Loss & Optimization
entity.setLearningRate(createReq.getLearningRate());
entity.setWeightDecay(createReq.getWeightDecay());
entity.setLayerDecayRate(createReq.getLayerDecayRate());
entity.setDdpFindUnusedParams(createReq.getDdpFindUnusedParams());
entity.setIgnoreIndex(createReq.getIgnoreIndex());
entity.setNumLayers(createReq.getNumLayers());
// Evaluation
entity.setMetrics(createReq.getMetrics());
entity.setSaveBest(createReq.getSaveBest());
entity.setSaveBestRule(createReq.getSaveBestRule());
entity.setValInterval(createReq.getValInterval());
entity.setLogInterval(createReq.getLogInterval());
entity.setVisInterval(createReq.getVisInterval());
// memo
entity.setMemo(createReq.getMemo());
}
/**
* 하이퍼파라미터 삭제
*
@@ -115,11 +82,16 @@ public class HyperParamCoreService {
.findHyperParamByUuid(uuid)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
if (entity.getHyperVer().equals("HPs_0001")) {
throw new CustomApiException("CONFLICT", HttpStatus.CONFLICT, "HPs_0001 버전은 삭제할수 없습니다.");
// if (entity.getHyperVer().equals("HPs_0001")) {
// throw new CustomApiException("UNPROCESSABLE_ENTITY", HttpStatus.UNPROCESSABLE_ENTITY);
// }
// 디폴트면 삭제불가
if (entity.getIsDefault()) {
throw new CustomApiException("UNPROCESSABLE_ENTITY", HttpStatus.UNPROCESSABLE_ENTITY);
}
entity.setDelYn("Y");
entity.setDelYn(true);
entity.setUpdatedUid(userUtil.getId());
entity.setUpdatedDttm(ZonedDateTime.now());
}
@@ -129,9 +101,12 @@ public class HyperParamCoreService {
*
* @return
*/
public HyperParamDto.Basic getInitHyperParam() {
ModelHyperParamEntity entity = new ModelHyperParamEntity();
entity.setUuid(null);
public HyperParamDto.Basic getInitHyperParam(ModelType model) {
ModelHyperParamEntity entity =
hyperParamRepository.getHyperParamByType(model).stream()
.filter(e -> e.getIsDefault() == Boolean.TRUE)
.findFirst()
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
return entity.toDto();
}
@@ -151,29 +126,91 @@ public class HyperParamCoreService {
/**
* 하이퍼파라미터 목록 조회
*
* @param model
* @param req
* @return
*/
public Page<HyperParamDto.List> findByHyperVerList(HyperParamDto.SearchReq req) {
return hyperParamRepository.findByHyperVerList(req);
public Page<HyperParamDto.List> findByHyperVerList(ModelType model, SearchReq req) {
return hyperParamRepository.findByHyperVerList(model, req);
}
/**
* 하이퍼파라미터 버전 조회
*
* @param model 모델 타입
* @return ver
*/
public String getFirstHyperParamVersion() {
public String getFirstHyperParamVersion(ModelType model) {
return hyperParamRepository
.findHyperParamVer()
.findHyperParamVerByModelType(model)
.map(ModelHyperParamEntity::getHyperVer)
.map(this::increase)
.orElse("HPs_0001");
.map(ver -> increase(ver, model))
.orElse(model.name() + "_000001");
}
private String increase(String hyperVer) {
String prefix = "HPs_";
/**
* 하이퍼 파라미터의 버전을 증가시킨다.
*
* @param hyperVer 현재 버전
* @param modelType 모델 타입
* @return 증가된 버전
*/
private String increase(String hyperVer, ModelType modelType) {
String prefix = modelType.name() + "_";
int num = Integer.parseInt(hyperVer.substring(prefix.length()));
return prefix + String.format("%04d", num + 1);
return prefix + String.format("%06d", num + 1);
}
private void applyHyperParam(ModelHyperParamEntity entity, HyperParam src) {
ModelType model = src.getModel();
// 하드코딩 모델별로 다른경우 250212 bbn 하드코딩
if (model == ModelType.G3) {
entity.setCropSize("512,512");
} else {
entity.setCropSize("256,256");
}
entity.setCropSize(src.getCropSize());
// Important
entity.setModelType(model); // 20250212 modeltype추가
entity.setBackbone(src.getBackbone());
entity.setInputSize(src.getInputSize());
entity.setBatchSize(src.getBatchSize());
// Data
entity.setTrainNumWorkers(src.getTrainNumWorkers());
entity.setValNumWorkers(src.getValNumWorkers());
entity.setTestNumWorkers(src.getTestNumWorkers());
entity.setTrainShuffle(src.getTrainShuffle());
entity.setTrainPersistent(src.getTrainPersistent());
entity.setValPersistent(src.getValPersistent());
// Model Architecture
entity.setDropPathRate(src.getDropPathRate());
entity.setFrozenStages(src.getFrozenStages());
entity.setNeckPolicy(src.getNeckPolicy());
entity.setClassWeight(src.getClassWeight());
entity.setDecoderChannels(src.getDecoderChannels());
// Loss & Optimization
entity.setLearningRate(src.getLearningRate());
entity.setWeightDecay(src.getWeightDecay());
entity.setLayerDecayRate(src.getLayerDecayRate());
entity.setDdpFindUnusedParams(src.getDdpFindUnusedParams());
entity.setIgnoreIndex(src.getIgnoreIndex());
entity.setNumLayers(src.getNumLayers());
// Evaluation
entity.setMetrics(src.getMetrics());
entity.setSaveBest(src.getSaveBest());
entity.setSaveBestRule(src.getSaveBestRule());
entity.setValInterval(src.getValInterval());
entity.setLogInterval(src.getLogInterval());
entity.setVisInterval(src.getVisInterval());
// memo
entity.setMemo(src.getMemo());
}
}

View File

@@ -1,197 +0,0 @@
package com.kamco.cd.training.postgres.core;
import com.kamco.cd.training.common.exception.BadRequestException;
import com.kamco.cd.training.common.exception.NotFoundException;
import com.kamco.cd.training.model.dto.ModelMngDto;
import com.kamco.cd.training.model.dto.ModelMngDto.Basic;
import com.kamco.cd.training.postgres.entity.ModelDatasetMappEntity;
import com.kamco.cd.training.postgres.entity.ModelTrainMasterEntity;
import com.kamco.cd.training.postgres.repository.model.ModelDatasetMappRepository;
import com.kamco.cd.training.postgres.repository.model.ModelMngRepository;
import java.time.ZonedDateTime;
import java.util.List;
import lombok.RequiredArgsConstructor;
import org.springframework.data.domain.Page;
import org.springframework.stereotype.Service;
@Service
@RequiredArgsConstructor
public class ModelMngCoreService {
private final ModelMngRepository modelMngRepository;
private final ModelDatasetMappRepository modelDatasetMappRepository;
/**
* 모델 목록 조회
*
* @param searchReq 검색 조건
* @return 페이징 처리된 모델 목록
*/
public Page<Basic> findByModels(ModelMngDto.SearchReq searchReq) {
Page<ModelTrainMasterEntity> entityPage = modelMngRepository.findByModels(searchReq);
return entityPage.map(ModelTrainMasterEntity::toDto);
}
/**
* 모델 상세 조회
*
* @param modelUid 모델 UID
* @return 모델 상세 정보
*/
public ModelMngDto.Detail getModelDetail(Long modelUid) {
ModelTrainMasterEntity entity =
modelMngRepository
.findById(modelUid)
.orElseThrow(() -> new NotFoundException("모델을 찾을 수 없습니다. ID: " + modelUid));
if (Boolean.TRUE.equals(entity.getDelYn())) {
throw new NotFoundException("삭제된 모델입니다. ID: " + modelUid);
}
return ModelMngDto.Detail.builder()
.uuid(entity.getUuid().toString())
.modelVer(entity.getModelVer())
.epochVer(entity.getEpochVer())
.processStep(entity.getProcessStep())
.trainStartDttm(entity.getTrainStartDttm())
.epochCnt(entity.getEpochCnt())
.datasetRatio(entity.getDatasetRatio())
.bestEpoch(entity.getBestEpoch())
.confirmedBestEpoch(entity.getConfirmedBestEpoch())
.step1EndDttm(entity.getStep1EndDttm())
.step1Duration(entity.getStep1Duration())
.step2EndDttm(entity.getStep2EndDttm())
.step2Duration(entity.getStep2Duration())
.progressRate(entity.getProgressRate())
.createdDttm(entity.getCreatedDttm())
.updatedDttm(entity.getUpdatedDttm())
.modelPath(entity.getModelPath())
.errorMsg(entity.getErrorMsg())
.build();
}
/**
* 모델 상세 조회 (UUID 기반)
*
* @param uuid 모델 UUID
* @return 모델 상세 정보
*/
public ModelMngDto.Detail getModelDetailByUuid(String uuid) {
ModelTrainMasterEntity entity = findByUuid(uuid);
return getModelDetail(entity.getId());
}
/**
* 학습 모델 전체 목록 조회 (삭제되지 않은 것만)
*
* @return 학습 모델 목록
*/
public List<ModelMngDto.TrainListRes> findAllTrainModels() {
List<ModelTrainMasterEntity> entities =
modelMngRepository.findByDelYnOrderByCreatedDttmDesc(false);
return entities.stream()
.map(
entity ->
ModelMngDto.TrainListRes.builder()
.uuid(entity.getUuid().toString())
.modelVer(entity.getModelVer())
.processStep(entity.getProcessStep())
.trainStartDttm(entity.getTrainStartDttm())
.progressRate(entity.getProgressRate())
.epochCnt(entity.getEpochCnt())
.step1EndDttm(entity.getStep1EndDttm())
.step1Duration(entity.getStep1Duration())
.step2EndDttm(entity.getStep2EndDttm())
.step2Duration(entity.getStep2Duration())
.createdDttm(entity.getCreatedDttm())
.errorMsg(entity.getErrorMsg())
.canResume(entity.getCanResume())
.lastCheckpointEpoch(entity.getLastCheckpointEpoch())
.build())
.toList();
}
/**
* 현재 실행 중인 모델 확인
*
* @return 실행 중인 모델 UUID (없으면 null)
*/
public String findRunningModelUuid() {
return modelMngRepository
.findFirstByStatusCdAndDelYn("RUNNING", false)
.map(entity -> entity.getUuid().toString())
.orElse(null);
}
/**
* 학습 마스터 생성
*
* @param trainReq 학습 시작 요청
* @return 생성된 모델 Entity
*/
public ModelTrainMasterEntity createTrainMaster(ModelMngDto.TrainStartReq trainReq) {
ModelTrainMasterEntity entity = new ModelTrainMasterEntity();
entity.setModelVer(trainReq.getHyperVer());
entity.setEpochVer(String.valueOf(trainReq.getEpoch()));
entity.setProcessStep("STEP1");
entity.setTrainStartDttm(ZonedDateTime.now());
entity.setEpochCnt(trainReq.getEpoch());
entity.setDatasetRatio(trainReq.getDatasetRatio());
entity.setDelYn(false);
entity.setCreatedDttm(ZonedDateTime.now());
entity.setProgressRate(0);
return modelMngRepository.save(entity);
}
/**
* 데이터셋 매핑 생성
*
* @param modelUid 모델 UID
* @param datasetIds 데이터셋 ID 목록
*/
public void createDatasetMappings(Long modelUid, List<Long> datasetIds) {
for (Long datasetId : datasetIds) {
ModelDatasetMappEntity mapping = new ModelDatasetMappEntity();
mapping.setModelUid(modelUid);
mapping.setDatasetUid(datasetId);
mapping.setDatasetType("TRAIN");
modelDatasetMappRepository.save(mapping);
}
}
/**
* UUID로 모델 조회
*
* @param uuid UUID
* @return 모델 Entity
*/
public ModelTrainMasterEntity findByUuid(String uuid) {
try {
java.util.UUID uuidObj = java.util.UUID.fromString(uuid);
return modelMngRepository
.findByUuid(uuidObj)
.orElseThrow(() -> new NotFoundException("모델을 찾을 수 없습니다. UUID: " + uuid));
} catch (IllegalArgumentException e) {
throw new BadRequestException("잘못된 UUID 형식입니다: " + uuid);
}
}
/**
* 모델 삭제 (논리 삭제)
*
* @param uuid UUID
*/
public void deleteByUuid(String uuid) {
ModelTrainMasterEntity entity = findByUuid(uuid);
// 진행 중인 모델은 삭제 불가
// if ("RUNNING".equals(entity.getStatusCd())) {
// throw new BadRequestException("진행 중인 모델은 삭제할 수 없습니다.");
// }
entity.setDelYn(true);
entity.setUpdatedDttm(ZonedDateTime.now());
modelMngRepository.save(entity);
}
}

View File

@@ -0,0 +1,50 @@
package com.kamco.cd.training.postgres.core;
import com.kamco.cd.training.postgres.repository.train.ModelTestMetricsJobRepository;
import com.kamco.cd.training.train.dto.ModelTrainMetricsDto.ModelMetricJsonDto;
import com.kamco.cd.training.train.dto.ModelTrainMetricsDto.ModelTestFileName;
import com.kamco.cd.training.train.dto.ModelTrainMetricsDto.ResponsePathDto;
import java.time.ZonedDateTime;
import java.util.List;
import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
@Service
@RequiredArgsConstructor
public class ModelTestMetricsJobCoreService {
private final ModelTestMetricsJobRepository modelTestMetricsJobRepository;
@Transactional
public void updateModelMetricsTrainSaveYn(Long modelId, String stepNo) {
modelTestMetricsJobRepository.updateModelMetricsTrainSaveYn(modelId, stepNo);
}
// Test 로직 시작
public List<ResponsePathDto> getTestMetricSaveNotYetModelIds() {
return modelTestMetricsJobRepository.getTestMetricSaveNotYetModelIds();
}
public void insertModelMetricsTest(List<Object[]> batchArgs) {
modelTestMetricsJobRepository.insertModelMetricsTest(batchArgs);
}
public ModelMetricJsonDto getTestMetricPackingInfo(Long modelId) {
return modelTestMetricsJobRepository.getTestMetricPackingInfo(modelId);
}
public ModelTestFileName findModelTestFileNames(Long modelId) {
return modelTestMetricsJobRepository.findModelTestFileNames(modelId);
}
@Transactional
public void updatePackingStart(Long modelId, ZonedDateTime now) {
modelTestMetricsJobRepository.updatePackingStart(modelId, now);
}
@Transactional
public void updatePackingEnd(Long modelId, ZonedDateTime now, String failSuccState) {
modelTestMetricsJobRepository.updatePackingEnd(modelId, now, failSuccState);
}
}

View File

@@ -0,0 +1,121 @@
package com.kamco.cd.training.postgres.core;
import com.kamco.cd.training.common.exception.BadRequestException;
import com.kamco.cd.training.common.exception.NotFoundException;
import com.kamco.cd.training.common.utils.UserUtil;
import com.kamco.cd.training.model.dto.ModelConfigDto;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.DetailSummary;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.HyperSummary;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.MappingDataset;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelBestEpoch;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelFileInfo;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelTestMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelTrainMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelValidationMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.TransferHyperSummary;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.Basic;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.ModelProgressStepDto;
import com.kamco.cd.training.postgres.entity.ModelMasterEntity;
import com.kamco.cd.training.postgres.repository.model.ModelConfigRepository;
import com.kamco.cd.training.postgres.repository.model.ModelDetailRepository;
import java.util.List;
import java.util.UUID;
import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Service;
@Service
@RequiredArgsConstructor
public class ModelTrainDetailCoreService {
private final ModelDetailRepository modelDetailRepository;
private final ModelConfigRepository modelConfigRepository;
private final UserUtil userUtil;
/**
* UUID로 모델 조회
*
* @param uuid UUID
* @return 모델 Entity
*/
public ModelMasterEntity findByUuid(String uuid) {
try {
UUID uuidObj = UUID.fromString(uuid);
return modelDetailRepository
.findByUuid(uuidObj)
.orElseThrow(() -> new NotFoundException("모델을 찾을 수 없습니다. UUID: " + uuid));
} catch (IllegalArgumentException e) {
throw new BadRequestException("잘못된 UUID 형식입니다: " + uuid);
}
}
/**
* 상세정보 페이지 > 요약정보
*
* @param uuid
* @return
*/
public DetailSummary getModelDetailSummary(UUID uuid) {
return modelDetailRepository.getModelDetailSummary(uuid);
}
/**
* 하이퍼 파리미터 요약정보
*
* @param uuid 모델마스터 uuid
* @return
*/
public HyperSummary getByModelHyperParamSummary(UUID uuid) {
return modelDetailRepository.getByModelHyperParamSummary(uuid);
}
public TransferHyperSummary getTransferHyperSummary(UUID uuid) {
return modelDetailRepository.getByModelTransferHyperParamSummary(uuid);
}
public List<MappingDataset> getByModelMappingDataset(UUID uuid) {
return modelDetailRepository.getByModelMappingDataset(uuid);
}
public Basic findByModelByUUID(UUID uuid) {
ModelMasterEntity entity = modelDetailRepository.findByModelByUUID(uuid);
return entity.toDto();
}
/**
* 모델 학습별 config 정보 조회
*
* @param modelId
* @return
*/
public ModelConfigDto.Basic findModelConfig(Long modelId) {
return modelConfigRepository.findModelConfigByModelId(modelId).orElse(null);
}
public List<ModelTrainMetrics> getModelTrainMetricResult(UUID uuid) {
return modelDetailRepository.getModelTrainMetricResult(uuid);
}
public List<ModelValidationMetrics> getModelValidationMetricResult(UUID uuid) {
return modelDetailRepository.getModelValidationMetricResult(uuid);
}
public List<ModelTestMetrics> getModelTestMetricResult(UUID uuid) {
return modelDetailRepository.getModelTestMetricResult(uuid);
}
public ModelBestEpoch getModelTrainBestEpoch(UUID uuid) {
return modelDetailRepository.getModelTrainBestEpoch(uuid);
}
public ModelFileInfo getModelTrainFileInfo(UUID uuid) {
return modelDetailRepository.getModelTrainFileInfo(uuid);
}
public List<ModelProgressStepDto> findModelTrainProgressInfo(UUID uuid) {
return modelDetailRepository.findModelTrainProgressInfo(uuid);
}
public Basic findByModelBeforeId(Long beforeModelId) {
ModelMasterEntity entity = modelDetailRepository.findByModelBeforeId(beforeModelId);
return entity.toDto();
}
}

View File

@@ -0,0 +1,187 @@
package com.kamco.cd.training.postgres.core;
import com.kamco.cd.training.common.exception.CustomApiException;
import com.kamco.cd.training.postgres.entity.ModelTrainJobEntity;
import com.kamco.cd.training.postgres.repository.train.ModelTrainJobRepository;
import com.kamco.cd.training.train.dto.ModelTrainJobDto;
import java.time.ZonedDateTime;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import lombok.RequiredArgsConstructor;
import lombok.extern.log4j.Log4j2;
import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
@Log4j2
@Service
@RequiredArgsConstructor
@Transactional(readOnly = true)
public class ModelTrainJobCoreService {
private final ModelTrainJobRepository modelTrainJobRepository;
public int findMaxAttemptNo(Long modelId) {
return modelTrainJobRepository.findMaxAttemptNo(modelId);
}
public Optional<ModelTrainJobDto> findLatestByModelId(Long modelId) {
return modelTrainJobRepository.findLatestByModelId(modelId).map(ModelTrainJobEntity::toDto);
}
public Optional<ModelTrainJobDto> findById(Long jobId) {
return modelTrainJobRepository.findById(jobId).map(ModelTrainJobEntity::toDto);
}
/** QUEUED Job 생성 */
@Transactional
public Long createQueuedJob(
Long modelId, int attemptNo, Map<String, Object> paramsJson, ZonedDateTime queuedDttm) {
ModelTrainJobEntity job = new ModelTrainJobEntity();
job.setModelId(modelId);
job.setAttemptNo(attemptNo);
job.setStatusCd("QUEUED");
job.setParamsJson(paramsJson);
job.setQueuedDttm(queuedDttm != null ? queuedDttm : ZonedDateTime.now());
modelTrainJobRepository.save(job);
modelTrainJobRepository.flush();
return job.getId();
}
/** 실행 시작 처리 */
@Transactional
public void markRunning(
Long jobId,
String containerName,
String logPath,
String lockedBy,
Integer totalEpoch,
String jobType) {
ModelTrainJobEntity job =
modelTrainJobRepository
.findById(jobId)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
job.setStatusCd("RUNNING");
job.setContainerName(containerName);
job.setLogPath(logPath);
job.setStartedDttm(ZonedDateTime.now());
job.setLockedDttm(ZonedDateTime.now());
job.setLockedBy(lockedBy);
job.setJobType(jobType);
if (totalEpoch != null) {
job.setTotalEpoch(totalEpoch);
}
}
/**
* 성공 처리
*
* @param jobId
* @param exitCode
*/
@Transactional
public void markSuccess(Long jobId, int exitCode) {
ModelTrainJobEntity job =
modelTrainJobRepository
.findById(jobId)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
job.setStatusCd("SUCCESS");
job.setExitCode(exitCode);
job.setFinishedDttm(ZonedDateTime.now());
}
/**
* 실패 처리
*
* @param jobId
* @param exitCode
* @param errorMessage
*/
@Transactional
public void markFailed(Long jobId, Integer exitCode, String errorMessage) {
ModelTrainJobEntity job =
modelTrainJobRepository
.findById(jobId)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
job.setStatusCd("FAILED");
job.setExitCode(exitCode);
job.setErrorMessage(errorMessage);
job.setFinishedDttm(ZonedDateTime.now());
log.info("[TRAIN JOB FAIL] jobId={}, modelId={}", jobId, errorMessage);
}
/**
* 중단됨 처리
*
* @param jobId
* @param exitCode
* @param errorMessage
*/
@Transactional
public void markPaused(Long jobId, Integer exitCode, String errorMessage) {
ModelTrainJobEntity job =
modelTrainJobRepository
.findById(jobId)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
job.setStatusCd("STOPPED");
job.setExitCode(exitCode);
job.setErrorMessage(errorMessage);
job.setFinishedDttm(ZonedDateTime.now());
log.info("[TRAIN JOB FAIL] jobId={}, modelId={}", jobId, errorMessage);
}
/** 취소 처리 */
@Transactional
public void markCanceled(Long jobId) {
ModelTrainJobEntity job =
modelTrainJobRepository
.findById(jobId)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
job.setStatusCd("STOPPED");
job.setFinishedDttm(ZonedDateTime.now());
}
@Transactional
public void updateEpoch(String containerName, Integer epoch) {
ModelTrainJobEntity job =
modelTrainJobRepository
.findByContainerName(containerName)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
job.setCurrentEpoch(epoch);
if (Objects.equals(job.getTotalEpoch(), epoch)) {}
}
public void insertModelTestTrainingRun(Long modelId, Long jobId, int epoch) {
modelTrainJobRepository.insertModelTestTrainingRun(modelId, jobId, epoch);
}
/**
* 실행중인 학습이 있는지 조회
*
* @return
*/
public List<ModelTrainJobDto> findRunningJobs() {
List<ModelTrainJobEntity> entity = modelTrainJobRepository.findRunningJobs();
if (entity == null || entity.isEmpty()) {
return Collections.emptyList();
}
return entity.stream().map(ModelTrainJobEntity::toDto).toList();
}
}

View File

@@ -0,0 +1,37 @@
package com.kamco.cd.training.postgres.core;
import com.kamco.cd.training.postgres.repository.train.ModelTrainMetricsJobRepository;
import com.kamco.cd.training.train.dto.ModelTrainMetricsDto.ResponsePathDto;
import java.util.List;
import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
@Service
@RequiredArgsConstructor
public class ModelTrainMetricsJobCoreService {
private final ModelTrainMetricsJobRepository modelTrainMetricsJobRepository;
public List<ResponsePathDto> getTrainMetricSaveNotYetModelIds() {
return modelTrainMetricsJobRepository.getTrainMetricSaveNotYetModelIds();
}
public void insertModelMetricsTrain(List<Object[]> batchArgs) {
modelTrainMetricsJobRepository.insertModelMetricsTrain(batchArgs);
}
@Transactional
public void updateModelMetricsTrainSaveYn(Long modelId, String stepNo) {
modelTrainMetricsJobRepository.updateModelMetricsTrainSaveYn(modelId, stepNo);
}
public void insertModelMetricsValidation(List<Object[]> batchArgs) {
modelTrainMetricsJobRepository.insertModelMetricsValidation(batchArgs);
}
@Transactional
public void updateModelSelectedBestEpoch(Long modelId, Integer epoch) {
modelTrainMetricsJobRepository.updateModelSelectedBestEpoch(modelId, epoch);
}
}

View File

@@ -0,0 +1,599 @@
package com.kamco.cd.training.postgres.core;
import com.kamco.cd.training.common.enums.HyperParamSelectType;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.common.enums.TrainStatusType;
import com.kamco.cd.training.common.exception.BadRequestException;
import com.kamco.cd.training.common.exception.CustomApiException;
import com.kamco.cd.training.common.utils.UserUtil;
import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetReq;
import com.kamco.cd.training.dataset.dto.DatasetDto.SelectDataSet;
import com.kamco.cd.training.dataset.dto.DatasetDto.SelectTransferDataSet;
import com.kamco.cd.training.model.dto.ModelConfigDto;
import com.kamco.cd.training.model.dto.ModelTrainMngDto;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.ListDto;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.TrainingDataset;
import com.kamco.cd.training.postgres.entity.ModelConfigEntity;
import com.kamco.cd.training.postgres.entity.ModelDatasetEntity;
import com.kamco.cd.training.postgres.entity.ModelDatasetMappEntity;
import com.kamco.cd.training.postgres.entity.ModelHyperParamEntity;
import com.kamco.cd.training.postgres.entity.ModelMasterEntity;
import com.kamco.cd.training.postgres.repository.dataset.DatasetRepository;
import com.kamco.cd.training.postgres.repository.hyperparam.HyperParamRepository;
import com.kamco.cd.training.postgres.repository.model.ModelConfigRepository;
import com.kamco.cd.training.postgres.repository.model.ModelDatasetMappRepository;
import com.kamco.cd.training.postgres.repository.model.ModelDatasetRepository;
import com.kamco.cd.training.postgres.repository.model.ModelMngRepository;
import com.kamco.cd.training.train.dto.ModelTrainLinkDto;
import com.kamco.cd.training.train.dto.TrainRunRequest;
import java.time.ZonedDateTime;
import java.util.ArrayList;
import java.util.List;
import java.util.UUID;
import lombok.RequiredArgsConstructor;
import org.springframework.data.domain.Page;
import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;
@Service
@RequiredArgsConstructor
public class ModelTrainMngCoreService {
private final ModelMngRepository modelMngRepository;
private final ModelDatasetRepository modelDatasetRepository;
private final ModelDatasetMappRepository modelDatasetMapRepository;
private final ModelConfigRepository modelConfigRepository;
private final HyperParamRepository hyperParamRepository;
private final DatasetRepository datasetRepository;
private final UserUtil userUtil;
/**
* 모델 목록 조회
*
* @param searchReq 검색 조건
* @return 페이징 처리된 모델 목록
*/
public Page<ListDto> findByModelList(ModelTrainMngDto.SearchReq searchReq) {
// Page<ModelMasterEntity> entityPage = modelMngRepository.findByModels(searchReq);
// return entityPage.map(ModelMasterEntity::toDto);
return modelMngRepository.findByModels(searchReq);
}
/**
* 학습모델 삭제
*
* @param uuid
*/
public void deleteModel(UUID uuid) {
ModelMasterEntity entity =
modelMngRepository
.findByUuid(uuid)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
entity.setDelYn(true);
entity.setUpdatedDttm(ZonedDateTime.now());
entity.setUpdatedUid(userUtil.getId());
}
/**
* 모델학습 저장
*
* @param addReq
* @return
*/
public ModelTrainMngDto.Basic saveModel(ModelTrainMngDto.AddReq addReq) {
ModelMasterEntity entity = new ModelMasterEntity();
ModelHyperParamEntity hyperParamEntity = new ModelHyperParamEntity();
// 최적화 파라미터는 모델 type의 디폴트사용
if (HyperParamSelectType.OPTIMIZED.getId().equals(addReq.getHyperParamType())) {
ModelType modelType = ModelType.getValueData(addReq.getModelNo());
hyperParamEntity =
hyperParamRepository.getHyperParamByType(modelType).stream()
.filter(e -> e.getIsDefault() == Boolean.TRUE)
.findFirst()
.orElse(null);
// hyperParamEntity = hyperParamRepository.findByHyperVer("HPs_0001").orElse(null);
} else {
hyperParamEntity =
hyperParamRepository.findHyperParamByUuid(addReq.getHyperUuid()).orElse(null);
}
if (hyperParamEntity == null || hyperParamEntity.getHyperVer() == null) {
throw new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND);
}
// 하이퍼 파라미터 사용 횟수 업데이트
hyperParamEntity.setTotalUseCnt(
hyperParamEntity.getTotalUseCnt() == null ? 1 : hyperParamEntity.getTotalUseCnt() + 1);
// 최근 사용일시 업데이트
hyperParamEntity.setLastUsedDttm(ZonedDateTime.now());
String modelVer =
String.join(
".", addReq.getModelNo(), hyperParamEntity.getHyperVer(), entity.getUuid().toString());
entity.setModelVer(modelVer);
entity.setHyperParamId(hyperParamEntity.getId());
entity.setModelNo(addReq.getModelNo());
entity.setTrainType(addReq.getTrainType()); // 일반, 전이
entity.setBeforeModelId(addReq.getBeforeModelId());
entity.setStatusCd(TrainStatusType.READY.getId());
entity.setStep1State(TrainStatusType.READY.getId());
entity.setCreatedUid(userUtil.getId());
ModelMasterEntity resultEntity = modelMngRepository.save(entity);
ModelTrainMngDto.Basic result = new ModelTrainMngDto.Basic();
result.setId(resultEntity.getId());
result.setUuid(resultEntity.getUuid());
return result;
}
/**
* data set 저장
*
* @param modelId 저장한 모델 학습 id
* @param addReq 요청 파라미터
*/
public void saveModelDataset(Long modelId, ModelTrainMngDto.AddReq addReq) {
TrainingDataset dataset = addReq.getTrainingDataset();
ModelMasterEntity modelMasterEntity = new ModelMasterEntity();
ModelDatasetEntity datasetEntity = new ModelDatasetEntity();
modelMasterEntity.setId(modelId);
datasetEntity.setModel(modelMasterEntity);
if (addReq.getModelNo().equals(ModelType.G1.getId())) {
datasetEntity.setBuildingCnt(dataset.getSummary().getBuildingCnt());
datasetEntity.setContainerCnt(dataset.getSummary().getContainerCnt());
} else if (addReq.getModelNo().equals(ModelType.G2.getId())) {
datasetEntity.setWasteCnt(dataset.getSummary().getWasteCnt());
} else if (addReq.getModelNo().equals(ModelType.G3.getId())) {
datasetEntity.setLandCoverCnt(dataset.getSummary().getLandCoverCnt());
}
datasetEntity.setCreatedUid(userUtil.getId());
// data set 저장
modelDatasetRepository.save(datasetEntity);
}
/**
* 학습모델 수정
*
* @param modelId
* @param req
*/
public void updateModelMaster(Long modelId, ModelTrainMngDto.UpdateReq req) {
ModelMasterEntity entity =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
// 임시폴더 UID업데이트
if (req.getRequestPath() != null && !req.getRequestPath().isEmpty()) {
entity.setRequestPath(req.getRequestPath());
}
}
/**
* 모델 데이터셋 mapping 테이블 저장
*
* @param modelId 모델학습 id
* @param datasetList 선택한 data set
*/
public void saveModelDatasetMap(Long modelId, List<Long> datasetList) {
for (Long datasetId : datasetList) {
ModelDatasetMappEntity mapEntity = new ModelDatasetMappEntity();
mapEntity.setModelUid(modelId);
mapEntity.setDatasetUid(datasetId);
modelDatasetMapRepository.save(mapEntity);
}
}
/**
* 모델학습 config 저장
*
* @param modelId 모델학습 id
* @param req 요청 파라미터
* @return
*/
public void saveModelConfig(Long modelId, ModelTrainMngDto.ModelConfig req) {
ModelMasterEntity modelMasterEntity = new ModelMasterEntity();
ModelConfigEntity entity = new ModelConfigEntity();
modelMasterEntity.setId(modelId);
entity.setModel(modelMasterEntity);
entity.setEpochCount(
req.getEpochCnt() < 10
? 10
: req.getEpochCnt()); // 에폭이 10 이하이면 10으로 고정하기. 10 이상 에폭으로 해야 best 에폭 파일이 생성되어 내려옴
entity.setTrainPercent(req.getTrainingCnt());
entity.setValidationPercent(req.getValidationCnt());
entity.setTestPercent(req.getTestCnt());
entity.setMemo(req.getMemo());
modelConfigRepository.save(entity);
}
/**
* 데이터셋 매핑 생성
*
* @param modelUid 모델 UID
* @param datasetIds 데이터셋 ID 목록
*/
public void createDatasetMappings(Long modelUid, List<Long> datasetIds) {
for (Long datasetId : datasetIds) {
ModelDatasetMappEntity mapping = new ModelDatasetMappEntity();
mapping.setModelUid(modelUid);
mapping.setDatasetUid(datasetId);
mapping.setDatasetType("TRAIN");
modelDatasetMapRepository.save(mapping);
}
}
/**
* UUID로 모델 조회
*
* @param uuid UUID
* @return 모델 Entity
*/
public ModelMasterEntity findByUuid(UUID uuid) {
try {
return modelMngRepository
.findByUuid(uuid)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
} catch (IllegalArgumentException e) {
throw new BadRequestException("잘못된 UUID 형식입니다: " + uuid);
}
}
/**
* uuid로 model id 조회
*
* @param uuid
* @return
*/
public Long findModelIdByUuid(UUID uuid) {
ModelMasterEntity entity =
modelMngRepository
.findByUuid(uuid)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
return entity.getId();
}
/**
* 모델학습 아이디로 config정보 조회
*
* @param uuid
* @return
*/
public ModelConfigDto.Basic findModelConfigByModelId(UUID uuid) {
ModelMasterEntity modelEntity = findByUuid(uuid);
return modelConfigRepository
.findModelConfigByModelId(modelEntity.getId())
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
}
public ModelConfigDto.TransferBasic findModelTransferConfigByModelId(UUID uuid) {
ModelMasterEntity modelEntity = findByUuid(uuid);
return modelConfigRepository
.findModelTransferConfigByModelId(modelEntity.getId())
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
}
/**
* 데이터셋 G1 목록
*
* @param req
* @return
*/
public List<SelectDataSet> getDatasetSelectG1List(DatasetReq req) {
return datasetRepository.getDatasetSelectG1List(req);
}
/**
* 전이학습 데이터셋 G1 목록
*
* @param modelId 모델 Id
* @return
*/
public List<SelectTransferDataSet> getDatasetTransferSelectG1List(Long modelId) {
return datasetRepository.getDatasetTransferSelectG1List(modelId);
}
/**
* 데이터셋 G2, G3 목록
*
* @param req
* @return
*/
public List<SelectDataSet> getDatasetSelectG2G3List(DatasetReq req) {
return datasetRepository.getDatasetSelectG2G3List(req);
}
/**
* 전이학습 데이터셋 G2, G3 목록
*
* @param modelId 모델 Id
* @param modelNo G2, G3
* @return
*/
public List<SelectTransferDataSet> getDatasetTransferSelectG2G3List(
Long modelId, String modelNo) {
return datasetRepository.getDatasetTransferSelectG2G3List(modelId, modelNo);
}
/**
* 모델관리 조회
*
* @param id
* @return
*/
public ModelTrainMngDto.Basic findModelById(Long id) {
ModelMasterEntity entity =
modelMngRepository
.findById(id)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + id));
return entity.toDto();
}
/** 마스터를 IN_PROGRESS로 전환하고, 현재 실행 jobId를 연결 - UI/중단/상태조회 모두 currentAttemptId를 기준으로 동작 */
@Transactional
public void markInProgress(Long modelId, Long jobId) {
ModelMasterEntity master =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
master.setStatusCd(TrainStatusType.IN_PROGRESS.getId());
master.setCurrentAttemptId(jobId);
// 필요하면 시작시간도 여기서 찍어줌
modelMngRepository.flush();
}
/** 마지막 에러 메시지 초기화 - 재시작/새 실행 때 이전 에러 흔적 제거 */
@Transactional
public void clearLastError(Long modelId) {
ModelMasterEntity master =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
master.setLastError(null);
modelMngRepository.flush();
}
/** 중단 처리(옵션) - cancel에서 쓰려고 하면 같이 구현 */
@Transactional
public void markStopped(Long modelId) {
ModelMasterEntity master =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
master.setStatusCd(TrainStatusType.STOPPED.getId());
}
/** 완료 처리(옵션) - Worker가 성공 시 호출 */
@Transactional
public void markCompleted(Long modelId) {
ModelMasterEntity master =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
master.setStatusCd(TrainStatusType.COMPLETED.getId());
}
/**
* step 1오류 처리(옵션) - Worker가 실패 시 호출
*
* @param modelId
* @param errorMessage
*/
@Transactional
public void markError(Long modelId, String errorMessage) {
ModelMasterEntity master =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
master.setStatusCd(TrainStatusType.ERROR.getId());
master.setStep1State(TrainStatusType.ERROR.getId());
master.setLastError(errorMessage);
master.setUpdatedUid(userUtil.getId());
master.setUpdatedDttm(ZonedDateTime.now());
}
/**
* step 2오류 처리(옵션) - Worker가 실패 시 호출
*
* @param modelId
* @param errorMessage
*/
@Transactional
public void markStep2Error(Long modelId, String errorMessage) {
ModelMasterEntity master =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
master.setStatusCd(TrainStatusType.ERROR.getId());
master.setStep2State(TrainStatusType.ERROR.getId());
master.setLastError(errorMessage);
master.setUpdatedUid(userUtil.getId());
master.setUpdatedDttm(ZonedDateTime.now());
}
@Transactional
public void markSuccess(Long modelId) {
ModelMasterEntity master =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
// 모델 상태 완료 처리
master.setStatusCd(TrainStatusType.COMPLETED.getId());
// (선택) 마지막 에러 메시지 비우기
master.setLastError(null);
}
/**
* 학습 실행에 필요한 파라미터 조회
*
* @param modelId
* @return
*/
public TrainRunRequest findTrainRunRequest(Long modelId) {
return modelMngRepository.findTrainRunRequest(modelId);
}
/**
* step1 진행중 처리
*
* @param modelId
* @param jobId
*/
@Transactional(propagation = Propagation.REQUIRES_NEW)
public void markStep1InProgress(Long modelId, Long jobId) {
ModelMasterEntity entity =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
entity.setStatusCd(TrainStatusType.IN_PROGRESS.getId());
entity.setStrtDttm(ZonedDateTime.now());
entity.setStep1StrtDttm(ZonedDateTime.now());
entity.setStep1State(TrainStatusType.IN_PROGRESS.getId());
entity.setCurrentAttemptId(jobId);
entity.setUpdatedDttm(ZonedDateTime.now());
entity.setUpdatedUid(userUtil.getId());
}
/**
* step2 진행중 처리
*
* @param modelId
*/
@Transactional(propagation = Propagation.REQUIRES_NEW)
public void markStep2InProgress(Long modelId, Long jobId) {
ModelMasterEntity entity =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
entity.setStatusCd(TrainStatusType.IN_PROGRESS.getId());
entity.setStep2StrtDttm(ZonedDateTime.now());
entity.setStep2State(TrainStatusType.IN_PROGRESS.getId());
entity.setCurrentAttemptId(jobId);
entity.setUpdatedDttm(ZonedDateTime.now());
entity.setUpdatedUid(userUtil.getId());
}
/**
* step1 완료처리
*
* @param modelId
*/
@Transactional(propagation = Propagation.REQUIRES_NEW)
public void markStep1Success(Long modelId) {
ModelMasterEntity entity =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
entity.setStatusCd(TrainStatusType.COMPLETED.getId());
entity.setStep1State(TrainStatusType.COMPLETED.getId());
entity.setStep1EndDttm(ZonedDateTime.now());
entity.setUpdatedDttm(ZonedDateTime.now());
entity.setUpdatedUid(userUtil.getId());
}
/**
* step2 완료처리
*
* @param modelId
*/
@Transactional(propagation = Propagation.REQUIRES_NEW)
public void markStep2Success(Long modelId) {
ModelMasterEntity entity =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
entity.setStatusCd(TrainStatusType.COMPLETED.getId());
entity.setStep2State(TrainStatusType.COMPLETED.getId());
entity.setStep2EndDttm(ZonedDateTime.now());
entity.setUpdatedDttm(ZonedDateTime.now());
entity.setUpdatedUid(userUtil.getId());
}
public void updateModelMasterBestEpoch(Long modelId, int epoch) {
ModelMasterEntity entity =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
entity.setBestEpoch(epoch);
}
/**
* 데이터셋 uid 조회
*
* @param datasetIds
* @return
*/
public List<String> findDatasetUid(List<Long> datasetIds) {
return datasetRepository.findDatasetUid(datasetIds);
}
public List<Long> findModelDatasetMapp(Long modelId) {
List<Long> datasetUids = new ArrayList<>();
List<ModelDatasetMappEntity> entities = modelDatasetMapRepository.findByModelUid(modelId);
for (ModelDatasetMappEntity entity : entities) {
datasetUids.add(entity.getDatasetUid());
}
return datasetUids;
}
public Long findModelStep1InProgressCnt() {
return modelMngRepository.findModelStep1InProgressCnt();
}
/**
* train 링크할 파일 경로
*
* @param modelId
* @return
*/
public List<ModelTrainLinkDto> findDatasetTrainPath(Long modelId) {
return modelDatasetMapRepository.findDatasetTrainPath(modelId);
}
/**
* validation 링크할 파일 경로
*
* @param modelId
* @return
*/
public List<ModelTrainLinkDto> findDatasetValPath(Long modelId) {
return modelDatasetMapRepository.findDatasetValPath(modelId);
}
/**
* test 링크할 파일 경로
*
* @param modelId
* @return
*/
public List<ModelTrainLinkDto> findDatasetTestPath(Long modelId) {
return modelDatasetMapRepository.findDatasetTestPath(modelId);
}
}

View File

@@ -2,6 +2,7 @@ package com.kamco.cd.training.postgres.core;
import com.kamco.cd.training.postgres.repository.upload.UploadSessionRepository;
import com.kamco.cd.training.upload.dto.UploadDto;
import java.util.UUID;
import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Service;
@@ -39,8 +40,8 @@ public class UploadSessionCoreService {
uploadSessionRepository.insertUploadSession(addReq);
}
public UploadDto.uploadDto findByDatasetUid(Long datasetId, String uploadDivi) {
return uploadSessionRepository.findByDatasetUid(datasetId, uploadDivi);
public UploadDto.uploadDto findByDatasetUid(String uploadDivi, UUID uuid) {
return uploadSessionRepository.findByDatasetUid(uploadDivi, uuid);
}
public UploadDto.uploadDto findByUuid(String uuid) {

View File

@@ -5,6 +5,7 @@ import com.kamco.cd.training.log.dto.EventStatus;
import com.kamco.cd.training.log.dto.EventType;
import com.kamco.cd.training.postgres.CommonCreateEntity;
import jakarta.persistence.*;
import java.util.UUID;
import lombok.AccessLevel;
import lombok.Getter;
import lombok.NoArgsConstructor;
@@ -14,6 +15,7 @@ import lombok.NoArgsConstructor;
@NoArgsConstructor(access = AccessLevel.PROTECTED)
@Table(name = "tb_audit_log")
public class AuditLogEntity extends CommonCreateEntity {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "audit_log_uid", nullable = false)
@@ -43,6 +45,12 @@ public class AuditLogEntity extends CommonCreateEntity {
@Column(name = "error_log_uid")
private Long errorLogUid;
@Column(name = "download_uuid")
private UUID downloadUuid;
@Column(name = "login_attempt_id")
private String loginAttemptId;
public AuditLogEntity(
Long userUid,
EventType eventType,
@@ -51,7 +59,9 @@ public class AuditLogEntity extends CommonCreateEntity {
String ipAddress,
String requestUri,
String requestBody,
Long errorLogUid) {
Long errorLogUid,
UUID downloadUuid,
String loginAttemptId) {
this.userUid = userUid;
this.eventType = eventType;
this.eventStatus = eventStatus;
@@ -60,6 +70,31 @@ public class AuditLogEntity extends CommonCreateEntity {
this.requestUri = requestUri;
this.requestBody = requestBody;
this.errorLogUid = errorLogUid;
this.downloadUuid = downloadUuid;
this.loginAttemptId = loginAttemptId;
}
/** 파일 다운로드 이력 생성 */
public static AuditLogEntity forFileDownload(
Long userId,
String requestUri,
String menuUid,
String ip,
int httpStatus,
UUID downloadUuid) {
return new AuditLogEntity(
userId,
EventType.DOWNLOAD, // 이벤트 타입 고정
httpStatus < 400 ? EventStatus.SUCCESS : EventStatus.FAILED, // 성공 여부
menuUid,
ip,
requestUri,
null, // requestBody 없음
null, // errorLogUid 없음
downloadUuid,
null // loginAttemptId 없음
);
}
public AuditLogDto.Basic toDto() {

View File

@@ -30,19 +30,14 @@ public class DatasetEntity {
@Column(name = "dataset_uid", nullable = false)
private Long id;
@Column(name = "uuid", nullable = false)
private UUID uuid;
@Column(name = "compare_yyyy", nullable = false)
private Integer compareYyyy;
@Size(max = 200)
@Column(name = "group_title", length = 200)
private String groupTitle;
@Size(max = 10)
@Column(name = "data_year", length = 10)
private String dataYear;
@Column(name = "target_yyyy", nullable = false)
private Integer targetYyyy;
@Size(max = 50)
@ColumnDefault("'CREATE'")
@ColumnDefault("'PRODUCTION'")
@Column(name = "data_type", length = 50)
private String dataType;
@@ -77,8 +72,9 @@ public class DatasetEntity {
@Column(name = "mod_dttm")
private Instant modDttm;
@Column(name = "id")
private Long id1;
@NotNull
@Column(name = "id", nullable = false)
private Long LegacyId;
@Size(max = 50)
@Column(name = "created_by", length = 50)
@@ -87,7 +83,7 @@ public class DatasetEntity {
@NotNull
@ColumnDefault("now()")
@Column(name = "created_dttm", nullable = false)
private ZonedDateTime createdDttm;
private ZonedDateTime createdDttm = ZonedDateTime.now();
@NotNull
@ColumnDefault("false")
@@ -108,25 +104,40 @@ public class DatasetEntity {
@Column(name = "updated_dttm")
private ZonedDateTime updatedDttm;
@Size(max = 4)
@Column(name = "year", length = 4)
private String year;
@NotNull
@ColumnDefault("uuid_generate_v4()")
@Column(name = "uuid", nullable = false)
private UUID uuid = UUID.randomUUID();
@Column(name = "class_counts", columnDefinition = "jsonb")
@ColumnDefault("0")
@Column(name = "total_object_count")
private Long totalObjectCount;
@Size(max = 1000)
@Column(name = "dataset_path", length = 1000)
private String datasetPath;
@Column(name = "class_counts")
@JdbcTypeCode(SqlTypes.JSON)
private Map<String, Integer> classCounts;
@Size(max = 32)
@Column(name = "uid")
private String uid;
public DatasetDto.Basic toDto() {
return new DatasetDto.Basic(
this.id,
this.uuid,
this.groupTitle,
this.title,
this.roundNo,
this.compareYyyy,
this.targetYyyy,
this.totalSize,
this.memo,
this.createdDttm,
this.status,
this.deleted);
this.deleted,
this.dataType);
}
}

View File

@@ -0,0 +1,118 @@
package com.kamco.cd.training.postgres.entity;
import com.kamco.cd.training.dataset.dto.DatasetObjDto;
import com.kamco.cd.training.dataset.dto.DatasetObjDto.Basic;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Size;
import java.time.ZonedDateTime;
import java.util.UUID;
import lombok.Getter;
import lombok.Setter;
import org.hibernate.annotations.ColumnDefault;
import org.hibernate.annotations.JdbcTypeCode;
import org.hibernate.type.SqlTypes;
import org.locationtech.jts.geom.Geometry;
@Getter
@Setter
@Entity
@Table(name = "tb_dataset_obj")
public class DatasetObjEntity {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "obj_id", nullable = false)
private Long objId;
@NotNull
@Column(name = "dataset_uid", nullable = false)
private Long datasetUid;
@Column(name = "target_yyyy")
private Integer targetYyyy;
@Size(max = 255)
@Column(name = "target_class_cd")
private String targetClassCd;
@Column(name = "compare_yyyy")
private Integer compareYyyy;
@Size(max = 255)
@Column(name = "compare_class_cd")
private String compareClassCd;
@Size(max = 255)
@Column(name = "target_path")
private String targetPath;
@Size(max = 255)
@Column(name = "compare_path")
private String comparePath;
@Size(max = 255)
@Column(name = "label_path")
private String labelPath;
@Size(max = 255)
@Column(name = "geojson_path")
private String geojsonPath;
@Size(max = 255)
@Column(name = "map_sheet_num")
private String mapSheetNum;
@ColumnDefault("now()")
@Column(name = "created_dttm")
private ZonedDateTime createdDttm;
@Column(name = "created_uid")
private Long createdUid;
@ColumnDefault("false")
@Column(name = "deleted")
private Boolean deleted;
@Column(name = "uuid")
private UUID uuid;
@Size(max = 32)
@Column(name = "uid")
private String uid;
@JdbcTypeCode(SqlTypes.JSON)
@Column(name = "geo_jsonb", columnDefinition = "jsonb")
private String geoJsonb;
@Column(name = "file_name")
private String fileName;
@Column(name = "geom", columnDefinition = "geometry")
private Geometry geom;
public Basic toDto() {
return new DatasetObjDto.Basic(
this.objId,
this.datasetUid,
this.targetYyyy,
this.targetClassCd,
this.compareYyyy,
this.compareClassCd,
this.targetPath,
this.comparePath,
this.labelPath,
this.geojsonPath,
this.mapSheetNum,
this.createdDttm,
this.createdUid,
this.deleted,
this.uuid,
this.geoJsonb);
}
}

View File

@@ -0,0 +1,117 @@
package com.kamco.cd.training.postgres.entity;
import com.kamco.cd.training.dataset.dto.DatasetObjDto.Basic;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Size;
import java.time.ZonedDateTime;
import java.util.UUID;
import lombok.Getter;
import lombok.Setter;
import org.hibernate.annotations.ColumnDefault;
import org.hibernate.annotations.JdbcTypeCode;
import org.hibernate.type.SqlTypes;
import org.locationtech.jts.geom.Geometry;
@Getter
@Setter
@Entity
@Table(name = "tb_dataset_test_obj")
public class DatasetTestObjEntity {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "obj_id", nullable = false)
private Long objId;
@NotNull
@Column(name = "dataset_uid", nullable = false)
private Long datasetUid;
@Column(name = "target_yyyy")
private Integer targetYyyy;
@Size(max = 255)
@Column(name = "target_class_cd")
private String targetClassCd;
@Column(name = "compare_yyyy")
private Integer compareYyyy;
@Size(max = 255)
@Column(name = "compare_class_cd")
private String compareClassCd;
@Size(max = 255)
@Column(name = "target_path")
private String targetPath;
@Size(max = 255)
@Column(name = "compare_path")
private String comparePath;
@Size(max = 255)
@Column(name = "label_path")
private String labelPath;
@Size(max = 255)
@Column(name = "geojson_path")
private String geojsonPath;
@Size(max = 255)
@Column(name = "map_sheet_num")
private String mapSheetNum;
@ColumnDefault("now()")
@Column(name = "created_dttm")
private ZonedDateTime createdDttm;
@Column(name = "created_uid")
private Long createdUid;
@ColumnDefault("false")
@Column(name = "deleted")
private Boolean deleted;
@Column(name = "uuid")
private UUID uuid;
@Size(max = 32)
@Column(name = "uid")
private String uid;
@JdbcTypeCode(SqlTypes.JSON)
@Column(name = "geo_jsonb", columnDefinition = "jsonb")
private String geoJsonb;
@Column(name = "file_name")
private String fileName;
@Column(name = "geom", columnDefinition = "geometry")
private Geometry geom;
public Basic toDto() {
return new Basic(
this.objId,
this.datasetUid,
this.targetYyyy,
this.targetClassCd,
this.compareYyyy,
this.compareClassCd,
this.targetPath,
this.comparePath,
this.labelPath,
this.geojsonPath,
this.mapSheetNum,
this.createdDttm,
this.createdUid,
this.deleted,
this.uuid,
this.geoJsonb);
}
}

View File

@@ -0,0 +1,117 @@
package com.kamco.cd.training.postgres.entity;
import com.kamco.cd.training.dataset.dto.DatasetObjDto.Basic;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Size;
import java.time.ZonedDateTime;
import java.util.UUID;
import lombok.Getter;
import lombok.Setter;
import org.hibernate.annotations.ColumnDefault;
import org.hibernate.annotations.JdbcTypeCode;
import org.hibernate.type.SqlTypes;
import org.locationtech.jts.geom.Geometry;
@Getter
@Setter
@Entity
@Table(name = "tb_dataset_val_obj")
public class DatasetValObjEntity {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "obj_id", nullable = false)
private Long objId;
@NotNull
@Column(name = "dataset_uid", nullable = false)
private Long datasetUid;
@Column(name = "target_yyyy")
private Integer targetYyyy;
@Size(max = 255)
@Column(name = "target_class_cd")
private String targetClassCd;
@Column(name = "compare_yyyy")
private Integer compareYyyy;
@Size(max = 255)
@Column(name = "compare_class_cd")
private String compareClassCd;
@Size(max = 255)
@Column(name = "target_path")
private String targetPath;
@Size(max = 255)
@Column(name = "compare_path")
private String comparePath;
@Size(max = 255)
@Column(name = "label_path")
private String labelPath;
@Size(max = 255)
@Column(name = "geojson_path")
private String geojsonPath;
@Size(max = 255)
@Column(name = "map_sheet_num")
private String mapSheetNum;
@ColumnDefault("now()")
@Column(name = "created_dttm")
private ZonedDateTime createdDttm;
@Column(name = "created_uid")
private Long createdUid;
@ColumnDefault("false")
@Column(name = "deleted")
private Boolean deleted;
@Column(name = "uuid")
private UUID uuid;
@Size(max = 32)
@Column(name = "uid")
private String uid;
@JdbcTypeCode(SqlTypes.JSON)
@Column(name = "geo_jsonb", columnDefinition = "jsonb")
private String geoJsonb;
@Column(name = "file_name")
private String fileName;
@Column(name = "geom", columnDefinition = "geometry")
private Geometry geom;
public Basic toDto() {
return new Basic(
this.objId,
this.datasetUid,
this.targetYyyy,
this.targetClassCd,
this.compareYyyy,
this.compareClassCd,
this.targetPath,
this.comparePath,
this.labelPath,
this.geojsonPath,
this.mapSheetNum,
this.createdDttm,
this.createdUid,
this.deleted,
this.uuid,
this.geoJsonb);
}
}

View File

@@ -0,0 +1,62 @@
package com.kamco.cd.training.postgres.entity;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.FetchType;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.JoinColumn;
import jakarta.persistence.ManyToOne;
import jakarta.persistence.Table;
import jakarta.validation.constraints.NotNull;
import java.time.ZonedDateTime;
import lombok.Getter;
import lombok.Setter;
import org.hibernate.annotations.ColumnDefault;
@Getter
@Setter
@Entity
@Table(name = "tb_model_config")
public class ModelConfigEntity {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "config_id", nullable = false)
private Long id;
@NotNull
@ManyToOne(fetch = FetchType.LAZY, optional = false)
@JoinColumn(name = "model_id", nullable = false)
private ModelMasterEntity model;
@Column(name = "epoch_count")
private Integer epochCount;
@Column(name = "train_percent")
private Float trainPercent;
@Column(name = "validation_percent")
private Float validationPercent;
@Column(name = "test_percent")
private Float testPercent;
@Column(name = "memo")
private String memo;
@ColumnDefault("now()")
@Column(name = "created_dttm")
private ZonedDateTime createdDttm;
@Column(name = "created_uid")
private Long createdUid;
@ColumnDefault("now()")
@Column(name = "updated_dttm")
private ZonedDateTime updatedDttm;
@Column(name = "updated_uid")
private Long updatedUid;
}

View File

@@ -0,0 +1,52 @@
package com.kamco.cd.training.postgres.entity;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.FetchType;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.JoinColumn;
import jakarta.persistence.ManyToOne;
import jakarta.persistence.Table;
import jakarta.validation.constraints.NotNull;
import java.time.ZonedDateTime;
import lombok.Getter;
import lombok.Setter;
import org.hibernate.annotations.ColumnDefault;
@Getter
@Setter
@Entity
@Table(name = "tb_model_dataset")
public class ModelDatasetEntity {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "id", nullable = false)
private Long id;
@NotNull
@ManyToOne(fetch = FetchType.LAZY, optional = false)
@JoinColumn(name = "model_id", nullable = false)
private ModelMasterEntity model;
@Column(name = "building_cnt")
private Long buildingCnt;
@Column(name = "container_cnt")
private Long containerCnt;
@Column(name = "waste_cnt")
private Long wasteCnt;
@Column(name = "land_cover_cnt")
private Long landCoverCnt;
@ColumnDefault("now()")
@Column(name = "created_dttm")
private ZonedDateTime createdDttm = ZonedDateTime.now();
@Column(name = "created_uid")
private Long createdUid;
}

View File

@@ -0,0 +1,74 @@
package com.kamco.cd.training.postgres.entity;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.FetchType;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.JoinColumn;
import jakarta.persistence.ManyToOne;
import jakarta.persistence.Table;
import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Size;
import java.time.ZonedDateTime;
import lombok.Getter;
import lombok.Setter;
import org.hibernate.annotations.ColumnDefault;
@Getter
@Setter
@Entity
@Table(name = "tb_model_history")
public class ModelHistoryEntity {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "hst_id", nullable = false)
private Integer id;
@NotNull
@ManyToOne(fetch = FetchType.LAZY, optional = false)
@JoinColumn(name = "model_id", nullable = false)
private ModelMasterEntity model;
@Column(name = "model_no")
private Short modelNo;
@Size(max = 50)
@Column(name = "model_ver", length = 50)
private String modelVer;
@Column(name = "model_step")
private Short modelStep;
@Column(name = "step1_strt_dttm")
private ZonedDateTime step1StrtDttm;
@Column(name = "step1_end_dttm")
private ZonedDateTime step1EndDttm;
@Size(max = 20)
@Column(name = "step1_state", length = 20)
private String step1State;
@Column(name = "step2_strt_dttm")
private ZonedDateTime step2StrtDttm;
@Column(name = "step2_end_dttm")
private ZonedDateTime step2EndDttm;
@Size(max = 20)
@Column(name = "step2_state", length = 20)
private String step2State;
@Column(name = "error_msg", length = Integer.MAX_VALUE)
private String errorMsg;
@ColumnDefault("now()")
@Column(name = "created_dttm")
private ZonedDateTime createdDttm;
@Column(name = "created_uid")
private Long createdUid;
}

View File

@@ -1,12 +1,11 @@
package com.kamco.cd.training.postgres.entity;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto;
import jakarta.persistence.*;
import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Size;
import java.time.ZonedDateTime;
import java.util.LinkedHashSet;
import java.util.Set;
import java.util.UUID;
import lombok.Getter;
import lombok.Setter;
@@ -193,10 +192,10 @@ public class ModelHyperParamEntity {
@Column(name = "save_best_rule", nullable = false, length = 10)
private String saveBestRule = "greater";
/** Default: 10 */
/** Default: 1 */
@NotNull
@Column(name = "val_interval", nullable = false)
private Integer valInterval = 10;
private Integer valInterval = 1;
/** Default: 400 */
@NotNull
@@ -280,9 +279,9 @@ public class ModelHyperParamEntity {
private String memo;
@NotNull
@ColumnDefault("'N'")
@ColumnDefault("false")
@Column(name = "del_yn", nullable = false, length = 1)
private String delYn = "N";
private Boolean delYn = false;
@NotNull
@ColumnDefault("CURRENT_TIMESTAMP")
@@ -304,22 +303,24 @@ public class ModelHyperParamEntity {
@Column(name = "last_used_dttm")
private ZonedDateTime lastUsedDttm;
@Column(name = "m1_use_cnt")
private Long m1UseCnt = 0L;
@Column(name = "model_type")
@Enumerated(EnumType.STRING)
private ModelType modelType;
@Column(name = "m2_use_cnt")
private Long m2UseCnt = 0L;
@Column(name = "default_param")
private Boolean isDefault = false;
@Column(name = "m3_use_cnt")
private Long m3UseCnt = 0L;
@OneToMany(mappedBy = "hyperParams", fetch = FetchType.LAZY)
private Set<ModelTrainMasterEntity> trainMasters = new LinkedHashSet<>();
@Column(name = "total_use_cnt")
private Integer totalUseCnt = 0;
public HyperParamDto.Basic toDto() {
return new HyperParamDto.Basic(
this.modelType,
this.uuid,
this.hyperVer,
this.createdDttm,
this.lastUsedDttm,
this.totalUseCnt,
// -------------------------
// Important
// -------------------------
@@ -389,6 +390,7 @@ public class ModelHyperParamEntity {
// -------------------------
this.gpuCnt,
this.gpuIds,
this.masterPort);
this.masterPort,
this.isDefault);
}
}

View File

@@ -0,0 +1,146 @@
package com.kamco.cd.training.postgres.entity;
import com.kamco.cd.training.model.dto.ModelTrainMngDto;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import jakarta.validation.constraints.Size;
import java.time.ZonedDateTime;
import java.util.UUID;
import lombok.Getter;
import lombok.Setter;
import org.hibernate.annotations.ColumnDefault;
@Getter
@Setter
@Entity
@Table(name = "tb_model_master")
public class ModelMasterEntity {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "model_id", nullable = false)
private Long id;
@Column(name = "hyper_param_id")
private Long hyperParamId;
@Size(max = 10)
@Column(name = "model_no", length = 10)
private String modelNo;
@Size(max = 200)
@Column(name = "model_ver", length = 200)
private String modelVer;
@Column(name = "model_step")
private Short modelStep;
@Size(max = 20)
@Column(name = "status_cd", length = 20)
private String statusCd;
@Column(name = "strt_dttm")
private ZonedDateTime strtDttm;
@Column(name = "step1_strt_dttm")
private ZonedDateTime step1StrtDttm;
@Column(name = "step1_end_dttm")
private ZonedDateTime step1EndDttm;
@Size(max = 20)
@Column(name = "step1_state", length = 20)
private String step1State;
@Column(name = "step2_strt_dttm")
private ZonedDateTime step2StrtDttm;
@Column(name = "step2_end_dttm")
private ZonedDateTime step2EndDttm;
@Size(max = 20)
@Column(name = "step2_state", length = 20)
private String step2State;
@Column(name = "del_yn")
private Boolean delYn = false;
@ColumnDefault("now()")
@Column(name = "created_dttm")
private ZonedDateTime createdDttm = ZonedDateTime.now();
@Column(name = "created_uid")
private Long createdUid;
@Column(name = "updated_dttm")
private ZonedDateTime updatedDttm;
@Column(name = "updated_uid")
private Long updatedUid;
@Column(name = "uuid")
private UUID uuid = UUID.randomUUID();
@Column(name = "train_type")
private String trainType;
@Column(name = "before_model_id")
private Long beforeModelId;
@Column(name = "step1_metric_save_yn")
private Boolean step1MetricSaveYn;
@Column(name = "step2_metric_save_yn")
private Boolean step2MetricSaveYn;
@Column(name = "current_attempt_id")
private Long currentAttemptId;
@Column(name = "last_error")
private String lastError;
@Column(name = "best_epoch")
private Integer bestEpoch;
@Column(name = "request_path")
private String requestPath;
@Column(name = "response_path")
private String responsePath;
@Column(name = "packing_state")
private String packingState;
@Column(name = "packing_strt_dttm")
private ZonedDateTime packingStrtDttm;
@Column(name = "packing_end_dttm")
private ZonedDateTime packingEndDttm;
public ModelTrainMngDto.Basic toDto() {
return new ModelTrainMngDto.Basic(
this.id,
this.uuid,
this.modelVer,
this.strtDttm,
this.step1StrtDttm,
this.step1EndDttm,
this.step2StrtDttm,
this.step2EndDttm,
this.step1State,
this.step2State,
this.statusCd,
this.trainType,
this.modelNo,
this.currentAttemptId,
this.requestPath,
this.packingState,
this.packingStrtDttm,
this.packingEndDttm,
this.beforeModelId);
}
}

View File

@@ -0,0 +1,75 @@
package com.kamco.cd.training.postgres.entity;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.FetchType;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.JoinColumn;
import jakarta.persistence.ManyToOne;
import jakarta.persistence.Table;
import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Size;
import java.time.ZonedDateTime;
import lombok.Getter;
import lombok.Setter;
import org.hibernate.annotations.ColumnDefault;
@Getter
@Setter
@Entity
@Table(name = "tb_model_metrics_test")
public class ModelMetricsTestEntity {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "mtrc_id", nullable = false)
private Long id;
@NotNull
@ManyToOne(fetch = FetchType.LAZY, optional = false)
@JoinColumn(name = "model_id", nullable = false)
private ModelMasterEntity model;
@Size(max = 50)
@Column(name = "model", length = 50)
private String model1;
@Column(name = "tp")
private Long tp;
@Column(name = "fp")
private Long fp;
@Column(name = "fn")
private Long fn;
@Column(name = "precisions")
private Float precisions;
@Column(name = "recall")
private Float recall;
@Column(name = "f1_score")
private Float f1Score;
@Column(name = "accuracy")
private Float accuracy;
@Column(name = "iou")
private Float iou;
@Column(name = "detection_count")
private Long detectionCount;
@Column(name = "gt_count")
private Long gtCount;
@ColumnDefault("now()")
@Column(name = "created_dttm")
private ZonedDateTime createdDttm;
@Column(name = "created_uid")
private Long createdUid;
}

View File

@@ -0,0 +1,55 @@
package com.kamco.cd.training.postgres.entity;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.FetchType;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.JoinColumn;
import jakarta.persistence.ManyToOne;
import jakarta.persistence.Table;
import jakarta.validation.constraints.NotNull;
import java.time.ZonedDateTime;
import lombok.Getter;
import lombok.Setter;
import org.hibernate.annotations.ColumnDefault;
@Getter
@Setter
@Entity
@Table(name = "tb_model_metrics_train")
public class ModelMetricsTrainEntity {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "mtrc_id", nullable = false)
private Long id;
@NotNull
@ManyToOne(fetch = FetchType.LAZY, optional = false)
@JoinColumn(name = "model_id", nullable = false)
private ModelMasterEntity model;
@Column(name = "epoch")
private Integer epoch;
@Column(name = "iteration")
private Long iteration;
@Column(name = "loss")
private Double loss;
@Column(name = "lr")
private Double lr;
@Column(name = "duration_time")
private Float durationTime;
@ColumnDefault("now()")
@Column(name = "created_dttm")
private ZonedDateTime createdDttm;
@Column(name = "created_uid")
private Long createdUid;
}

View File

@@ -0,0 +1,79 @@
package com.kamco.cd.training.postgres.entity;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.FetchType;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.JoinColumn;
import jakarta.persistence.ManyToOne;
import jakarta.persistence.Table;
import jakarta.validation.constraints.NotNull;
import java.time.ZonedDateTime;
import lombok.Getter;
import lombok.Setter;
import org.hibernate.annotations.ColumnDefault;
@Getter
@Setter
@Entity
@Table(name = "tb_model_metrics_validation")
public class ModelMetricsValidationEntity {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "mtrc_id", nullable = false)
private Long id;
@NotNull
@ManyToOne(fetch = FetchType.LAZY, optional = false)
@JoinColumn(name = "model_id", nullable = false)
private ModelMasterEntity model;
@Column(name = "epoch")
private Integer epoch;
@Column(name = "a_acc")
private Float aAcc;
@Column(name = "m_fscore")
private Float mFscore;
@Column(name = "m_precision")
private Float mPrecision;
@Column(name = "m_recall")
private Float mRecall;
@Column(name = "m_iou")
private Float mIou;
@Column(name = "m_acc")
private Float mAcc;
@Column(name = "changed_fscore")
private Float changedFscore;
@Column(name = "changed_precision")
private Float changedPrecision;
@Column(name = "changed_recall")
private Float changedRecall;
@Column(name = "unchanged_fscore")
private Float unchangedFscore;
@Column(name = "unchanged_precision")
private Float unchangedPrecision;
@Column(name = "unchanged_recall")
private Float unchangedRecall;
@ColumnDefault("now()")
@Column(name = "created_dttm")
private ZonedDateTime createdDttm;
@Column(name = "created_uid")
private Long createdUid;
}

View File

@@ -0,0 +1,42 @@
package com.kamco.cd.training.postgres.entity;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import jakarta.validation.constraints.NotNull;
import java.time.OffsetDateTime;
import lombok.Getter;
import lombok.Setter;
import org.hibernate.annotations.ColumnDefault;
@Getter
@Setter
@Entity
@Table(name = "tb_model_test_training_run")
public class ModelTestTrainingRunEntity {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "tsr_id", nullable = false)
private Long id;
@NotNull
@Column(name = "model_id", nullable = false)
private Long modelId;
@Column(name = "attempt_no")
private Integer attemptNo;
@Column(name = "job_id")
private Long jobId;
@Column(name = "epoch")
private Integer epoch;
@ColumnDefault("now()")
@Column(name = "created_dttm")
private OffsetDateTime createdDttm;
}

View File

@@ -0,0 +1,105 @@
package com.kamco.cd.training.postgres.entity;
import com.kamco.cd.training.train.dto.ModelTrainJobDto;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Size;
import java.time.ZonedDateTime;
import java.util.Map;
import lombok.Getter;
import lombok.Setter;
import org.hibernate.annotations.ColumnDefault;
import org.hibernate.annotations.JdbcTypeCode;
import org.hibernate.type.SqlTypes;
@Getter
@Setter
@Entity
@Table(name = "tb_model_train_job")
public class ModelTrainJobEntity {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "id", nullable = false)
private Long id;
@NotNull
@Column(name = "model_id", nullable = false)
private Long modelId;
@NotNull
@Column(name = "attempt_no", nullable = false)
private Integer attemptNo;
@Size(max = 30)
@NotNull
@Column(name = "status_cd", nullable = false, length = 30)
private String statusCd;
@NotNull
@Column(name = "params_json", nullable = false)
@JdbcTypeCode(SqlTypes.JSON)
private Map<String, Object> paramsJson;
@Size(max = 200)
@Column(name = "container_name", length = 200)
private String containerName;
@Size(max = 500)
@Column(name = "log_path", length = 500)
private String logPath;
@Column(name = "exit_code")
private Integer exitCode;
@Column(name = "error_message", columnDefinition = "TEXT")
private String errorMessage;
@ColumnDefault("now()")
@Column(name = "queued_dttm")
private ZonedDateTime queuedDttm;
@Column(name = "started_dttm")
private ZonedDateTime startedDttm;
@Column(name = "finished_dttm")
private ZonedDateTime finishedDttm;
@Column(name = "locked_dttm")
private ZonedDateTime lockedDttm;
@Size(max = 100)
@Column(name = "locked_by", length = 100)
private String lockedBy;
@Column(name = "total_epoch")
private Integer totalEpoch;
@Column(name = "current_epoch")
private Integer currentEpoch;
@Column(name = "job_type")
private String jobType;
public ModelTrainJobDto toDto() {
return new ModelTrainJobDto(
this.id,
this.modelId,
this.attemptNo,
this.statusCd,
this.exitCode,
this.errorMessage,
this.containerName,
this.paramsJson,
this.queuedDttm,
this.startedDttm,
this.finishedDttm,
this.totalEpoch,
this.currentEpoch);
}
}

View File

@@ -1,191 +0,0 @@
package com.kamco.cd.training.postgres.entity;
import com.kamco.cd.training.model.dto.ModelMngDto;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.FetchType;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.JoinColumn;
import jakarta.persistence.ManyToOne;
import jakarta.persistence.Table;
import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Size;
import java.time.ZonedDateTime;
import java.util.UUID;
import lombok.Getter;
import lombok.Setter;
import org.hibernate.annotations.ColumnDefault;
import org.hibernate.annotations.OnDelete;
import org.hibernate.annotations.OnDeleteAction;
import org.hibernate.annotations.UuidGenerator;
@Getter
@Setter
@Entity
@Table(name = "tb_model_train_master")
public class ModelTrainMasterEntity {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "model_uid", nullable = false)
private Long id;
@Size(max = 50)
@NotNull
@Column(name = "model_ver", nullable = false, length = 50)
private String modelVer;
@Size(max = 50)
@Column(name = "hyper_ver", length = 50)
private String hyperVer;
/** tb_model_train_master.hyper_param_id -> tb_model_hyper_params.hyper_param_id */
@NotNull
@Column(name = "hyper_param_id", nullable = false)
private Long hyperParamId;
/** - hyperParamId 컬럼으로 저장/수정하고 - 객체로 조회할 때만 hyperParams 사용 */
@ManyToOne(fetch = FetchType.LAZY)
@JoinColumn(
name = "hyper_param_id",
referencedColumnName = "hyper_param_id",
insertable = false,
updatable = false)
private ModelHyperParamEntity hyperParams;
@Size(max = 50)
@Column(name = "epoch_ver", length = 50)
private String epochVer;
@Size(max = 50)
@Column(name = "process_step", length = 50)
private String processStep;
@Column(name = "train_start_dttm")
private ZonedDateTime trainStartDttm;
@Column(name = "epoch_cnt")
private Integer epochCnt;
@Size(max = 50)
@Column(name = "dataset_ratio", length = 50)
private String datasetRatio;
@Column(name = "best_epoch")
private Integer bestEpoch;
@Column(name = "step1_end_dttm")
private ZonedDateTime step1EndDttm;
@Column(name = "step1_duration")
private ZonedDateTime step1Duration;
@Column(name = "step2_end_dttm")
private ZonedDateTime step2EndDttm;
@Column(name = "step2_duration")
private ZonedDateTime step2Duration;
@NotNull
@ColumnDefault("false")
@Column(name = "del_yn", nullable = false)
private Boolean delYn = false;
@Column(name = "created_uid")
private Long createdUid;
@Column(name = "updated_uid")
private Long updatedUid;
@ColumnDefault("now()")
@Column(name = "created_dttm")
private ZonedDateTime createdDttm;
@Column(name = "updated_dttm")
private ZonedDateTime updatedDttm;
@ColumnDefault("0")
@Column(name = "progress_rate")
private Integer progressRate;
@Column(name = "stop_dttm")
private ZonedDateTime stopDttm;
@Column(name = "confirmed_best_epoch")
private Integer confirmedBestEpoch;
@Size(max = 255)
@Column(name = "model_path")
private String modelPath;
@Column(name = "error_msg")
private String errorMsg;
@Column(name = "step2_start_dttm")
private ZonedDateTime step2StartDttm;
@Size(max = 1000)
@Column(name = "train_log_path", length = 1000)
private String trainLogPath;
@Column(name = "memo")
private String memo;
/** 기존 자기참조(base_model_uid)는 유지 */
@ManyToOne(fetch = FetchType.LAZY)
@OnDelete(action = OnDeleteAction.SET_NULL)
@JoinColumn(name = "base_model_uid")
private ModelTrainMasterEntity baseModel;
@Size(max = 1000)
@Column(name = "pretrained_model_path", length = 1000)
private String pretrainedModelPath;
@Column(name = "last_checkpoint_epoch")
private Integer lastCheckpointEpoch;
@Size(max = 500)
@Column(name = "checkpoint_path", length = 500)
private String checkpointPath;
@Column(name = "can_resume")
private Boolean canResume;
@NotNull
@UuidGenerator
@Column(name = "uuid", nullable = false, updatable = false)
private UUID uuid = UUID.randomUUID();
@Size(max = 10)
@Column(name = "step1_status", length = 10)
private String step1Status;
@Size(max = 10)
@Column(name = "step2_status", length = 10)
private String step2Status;
@Size(max = 10)
@Column(name = "transfer_status", length = 10)
private String transferStatus;
@Size(max = 10)
@Column(name = "status_cd", length = 10)
private String statusCd;
public ModelMngDto.Basic toDto() {
return new ModelMngDto.Basic(
this.id,
this.uuid,
this.modelVer,
this.trainStartDttm,
this.step1EndDttm,
this.step1Duration,
this.step2EndDttm,
this.step2Duration,
this.step1Status,
this.step2Status,
this.transferStatus);
}
}

View File

@@ -0,0 +1,7 @@
package com.kamco.cd.training.postgres.repository.dataset;
import com.kamco.cd.training.postgres.entity.DatasetObjEntity;
import org.springframework.data.jpa.repository.JpaRepository;
public interface DatasetObjRepository
extends JpaRepository<DatasetObjEntity, Long>, DatasetObjRepositoryCustom {}

View File

@@ -0,0 +1,27 @@
package com.kamco.cd.training.postgres.repository.dataset;
import com.kamco.cd.training.dataset.dto.DatasetObjDto.DatasetClass;
import com.kamco.cd.training.dataset.dto.DatasetObjDto.DatasetObjRegDto;
import com.kamco.cd.training.dataset.dto.DatasetObjDto.SearchReq;
import com.kamco.cd.training.postgres.entity.DatasetObjEntity;
import java.util.List;
import java.util.Optional;
import java.util.UUID;
import org.springframework.data.domain.Page;
public interface DatasetObjRepositoryCustom {
Optional<DatasetObjEntity> findByUuid(UUID id);
Page<DatasetObjEntity> searchDatasetObjectList(SearchReq searchReq);
List<DatasetClass> findDatasetObjClassByUuid(UUID uuid, String type);
void insertDatasetObj(DatasetObjRegDto objRegDto);
String getFilePathByUUIDPathType(UUID uuid, String pathType);
void insertDatasetTestObj(DatasetObjRegDto objRegDto);
void insertDatasetValObj(DatasetObjRegDto objRegDto);
}

View File

@@ -0,0 +1,279 @@
package com.kamco.cd.training.postgres.repository.dataset;
import static com.kamco.cd.training.postgres.entity.QDatasetEntity.datasetEntity;
import static com.kamco.cd.training.postgres.entity.QDatasetObjEntity.datasetObjEntity;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.kamco.cd.training.dataset.dto.DatasetObjDto.DatasetClass;
import com.kamco.cd.training.dataset.dto.DatasetObjDto.DatasetObjRegDto;
import com.kamco.cd.training.dataset.dto.DatasetObjDto.SearchReq;
import com.kamco.cd.training.postgres.entity.DatasetEntity;
import com.kamco.cd.training.postgres.entity.DatasetObjEntity;
import com.kamco.cd.training.postgres.entity.QDatasetEntity;
import com.querydsl.core.BooleanBuilder;
import com.querydsl.core.types.Expression;
import com.querydsl.core.types.Projections;
import com.querydsl.core.types.dsl.Expressions;
import com.querydsl.core.types.dsl.StringPath;
import com.querydsl.jpa.impl.JPAQueryFactory;
import jakarta.persistence.EntityManager;
import jakarta.persistence.EntityNotFoundException;
import jakarta.persistence.PersistenceContext;
import java.util.List;
import java.util.Objects;
import java.util.Optional;
import java.util.UUID;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.locationtech.jts.geom.Geometry;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.PageImpl;
import org.springframework.data.domain.Pageable;
import org.springframework.stereotype.Repository;
@Slf4j
@Repository
@RequiredArgsConstructor
public class DatasetObjRepositoryImpl implements DatasetObjRepositoryCustom {
private final JPAQueryFactory queryFactory;
private final QDatasetEntity dataset = datasetEntity;
@PersistenceContext EntityManager em;
@Override
public Optional<DatasetObjEntity> findByUuid(UUID id) {
return Optional.ofNullable(
queryFactory
.select(datasetObjEntity)
.from(datasetObjEntity)
.where(datasetObjEntity.uuid.eq(id), datasetObjEntity.deleted.isFalse())
.fetchOne());
}
@Override
public void insertDatasetTestObj(DatasetObjRegDto objRegDto) {
ObjectMapper objectMapper = new ObjectMapper();
String json;
Geometry geometry;
String geometryJson;
try {
json = objectMapper.writeValueAsString(objRegDto.getGeojson());
geometryJson =
objectMapper.writeValueAsString(
objRegDto.getGeojson().path("features").get(0).path("geometry"));
} catch (JsonProcessingException e) {
throw new RuntimeException(e);
}
try {
em.createNativeQuery(
"""
insert into tb_dataset_test_obj
(dataset_uid, target_yyyy, target_class_cd,
compare_yyyy, compare_class_cd,
target_path, compare_path, label_path, geo_jsonb, map_sheet_num, file_name, geom, geojson_path)
values
(?, ?, ?, ?, ?, ?, ?, ?, cast(? as jsonb), ?, ?, ST_SetSRID(ST_GeomFromGeoJSON(?), 5186), ?)
""")
.setParameter(1, objRegDto.getDatasetUid())
.setParameter(2, objRegDto.getTargetYyyy())
.setParameter(3, objRegDto.getTargetClassCd())
.setParameter(4, objRegDto.getCompareYyyy())
.setParameter(5, objRegDto.getCompareClassCd())
.setParameter(6, objRegDto.getTargetPath())
.setParameter(7, objRegDto.getComparePath())
.setParameter(8, objRegDto.getLabelPath())
.setParameter(9, json)
.setParameter(10, objRegDto.getMapSheetNum())
.setParameter(11, objRegDto.getFileName())
.setParameter(12, geometryJson)
.setParameter(13, objRegDto.getGeojsonPath())
.executeUpdate();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
@Override
public void insertDatasetValObj(DatasetObjRegDto objRegDto) {
ObjectMapper objectMapper = new ObjectMapper();
String json;
String geometryJson;
try {
json = objectMapper.writeValueAsString(objRegDto.getGeojson());
geometryJson =
objectMapper.writeValueAsString(
objRegDto.getGeojson().path("features").get(0).path("geometry"));
} catch (JsonProcessingException e) {
throw new RuntimeException(e);
}
try {
em.createNativeQuery(
"""
insert into tb_dataset_val_obj
(dataset_uid, target_yyyy, target_class_cd,
compare_yyyy, compare_class_cd,
target_path, compare_path, label_path, geo_jsonb, map_sheet_num, file_name, geom, geojson_path)
values
(?, ?, ?, ?, ?, ?, ?, ?, cast(? as jsonb), ?, ?, ST_SetSRID(ST_GeomFromGeoJSON(?), 5186), ?)
""")
.setParameter(1, objRegDto.getDatasetUid())
.setParameter(2, objRegDto.getTargetYyyy())
.setParameter(3, objRegDto.getTargetClassCd())
.setParameter(4, objRegDto.getCompareYyyy())
.setParameter(5, objRegDto.getCompareClassCd())
.setParameter(6, objRegDto.getTargetPath())
.setParameter(7, objRegDto.getComparePath())
.setParameter(8, objRegDto.getLabelPath())
.setParameter(9, json)
.setParameter(10, objRegDto.getMapSheetNum())
.setParameter(11, objRegDto.getFileName())
.setParameter(12, geometryJson)
.setParameter(13, objRegDto.getGeojsonPath())
.executeUpdate();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
@Override
public Page<DatasetObjEntity> searchDatasetObjectList(SearchReq searchReq) {
Pageable pageable = searchReq.toPageable();
BooleanBuilder builder = new BooleanBuilder();
if (StringUtils.isNotBlank(searchReq.getCompareClassCd())) {
builder.and(datasetObjEntity.compareClassCd.eq(searchReq.getCompareClassCd()));
}
if (StringUtils.isNotBlank(searchReq.getTargetClassCd())) {
builder.and(datasetObjEntity.targetClassCd.eq(searchReq.getTargetClassCd()));
}
if (StringUtils.isNotBlank(searchReq.getMapSheetNum())) {
builder.and(datasetObjEntity.mapSheetNum.eq(searchReq.getMapSheetNum()));
}
DatasetEntity entity =
queryFactory
.selectFrom(datasetEntity)
.where(datasetEntity.uuid.eq(searchReq.getUuid()))
.fetchOne();
if (Objects.isNull(entity)) {
throw new EntityNotFoundException("DatasetEntity not found for uuid: " + searchReq.getUuid());
}
List<DatasetObjEntity> content =
queryFactory
.selectFrom(datasetObjEntity)
.where(
datasetObjEntity
.deleted
.isFalse()
.and(datasetObjEntity.datasetUid.eq(entity.getId()))
.and(builder))
.offset(pageable.getOffset())
.limit(pageable.getPageSize())
.fetch();
// Count 쿼리 별도 실행 (null safe handling)
long total =
Optional.ofNullable(
queryFactory
.select(datasetObjEntity.count())
.from(datasetObjEntity)
.where(
datasetObjEntity
.deleted
.isFalse()
.and(datasetObjEntity.datasetUid.eq(entity.getId()))
.and(builder))
.fetchOne())
.orElse(0L);
return new PageImpl<>(content, pageable, total);
}
@Override
public List<DatasetClass> findDatasetObjClassByUuid(UUID uuid, String type) {
StringPath classCd =
type.equals("compare") ? datasetObjEntity.compareClassCd : datasetObjEntity.targetClassCd;
BooleanBuilder builder = new BooleanBuilder();
builder.and(datasetEntity.uuid.eq(uuid));
builder.and(datasetEntity.deleted.isFalse());
builder.and(datasetObjEntity.deleted.isFalse());
return queryFactory
.select(Projections.constructor(DatasetClass.class, classCd))
.from(datasetEntity)
.join(datasetObjEntity)
.on(datasetEntity.id.eq(datasetObjEntity.datasetUid))
.where(builder)
.groupBy(classCd)
.fetch();
}
@Override
public void insertDatasetObj(DatasetObjRegDto objRegDto) {
ObjectMapper objectMapper = new ObjectMapper();
String json;
String geometryJson;
try {
json = objectMapper.writeValueAsString(objRegDto.getGeojson());
geometryJson =
objectMapper.writeValueAsString(
objRegDto.getGeojson().path("features").get(0).path("geometry"));
} catch (JsonProcessingException e) {
throw new RuntimeException(e);
}
try {
em.createNativeQuery(
"""
insert into tb_dataset_obj
(dataset_uid, target_yyyy, target_class_cd,
compare_yyyy, compare_class_cd,
target_path, compare_path, label_path, geo_jsonb, map_sheet_num, file_name, geom, geojson_path)
values
(?, ?, ?, ?, ?, ?, ?, ?, cast(? as jsonb), ?, ?, ST_SetSRID(ST_GeomFromGeoJSON(?), 5186), ?)
""")
.setParameter(1, objRegDto.getDatasetUid())
.setParameter(2, objRegDto.getTargetYyyy())
.setParameter(3, objRegDto.getTargetClassCd())
.setParameter(4, objRegDto.getCompareYyyy())
.setParameter(5, objRegDto.getCompareClassCd())
.setParameter(6, objRegDto.getTargetPath())
.setParameter(7, objRegDto.getComparePath())
.setParameter(8, objRegDto.getLabelPath())
.setParameter(9, json)
.setParameter(10, objRegDto.getMapSheetNum())
.setParameter(11, objRegDto.getFileName())
.setParameter(12, geometryJson)
.setParameter(13, objRegDto.getGeojsonPath())
.executeUpdate();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
@Override
public String getFilePathByUUIDPathType(UUID uuid, String pathType) {
Expression<String> pathExpr =
switch (pathType) {
case "compare" -> datasetObjEntity.comparePath;
case "target" -> datasetObjEntity.targetPath;
case "label" -> datasetObjEntity.labelPath;
default -> Expressions.constant("");
};
return queryFactory
.select(pathExpr)
.from(datasetObjEntity)
.where(datasetObjEntity.uuid.eq(uuid))
.fetchOne();
}
}

View File

@@ -1,7 +1,12 @@
package com.kamco.cd.training.postgres.repository.dataset;
import com.kamco.cd.training.dataset.dto.DatasetDto;
import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetMngRegDto;
import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetReq;
import com.kamco.cd.training.dataset.dto.DatasetDto.SelectDataSet;
import com.kamco.cd.training.dataset.dto.DatasetDto.SelectTransferDataSet;
import com.kamco.cd.training.postgres.entity.DatasetEntity;
import java.util.List;
import java.util.Optional;
import java.util.UUID;
import org.springframework.data.domain.Page;
@@ -10,4 +15,20 @@ public interface DatasetRepositoryCustom {
Page<DatasetEntity> findDatasetList(DatasetDto.SearchReq searchReq);
Optional<DatasetEntity> findByUuid(UUID id);
List<SelectDataSet> getDatasetSelectG1List(DatasetReq req);
public List<SelectTransferDataSet> getDatasetTransferSelectG1List(Long modelId);
public List<SelectTransferDataSet> getDatasetTransferSelectG2G3List(Long modelId, String modelNo);
List<SelectDataSet> getDatasetSelectG2G3List(DatasetReq req);
Long getDatasetMaxStage(int compareYyyy, int targetYyyy);
Long insertDatasetMngData(DatasetMngRegDto mngRegDto);
List<String> findDatasetUid(List<Long> datasetIds);
Long findDatasetByUidExistsCnt(String uid);
}

View File

@@ -1,9 +1,25 @@
package com.kamco.cd.training.postgres.repository.dataset;
import com.kamco.cd.training.dataset.dto.DatasetDto;
import static com.kamco.cd.training.postgres.entity.QDatasetObjEntity.datasetObjEntity;
import static com.kamco.cd.training.postgres.entity.QModelDatasetMappEntity.modelDatasetMappEntity;
import static com.kamco.cd.training.postgres.entity.QModelMasterEntity.modelMasterEntity;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetMngRegDto;
import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetReq;
import com.kamco.cd.training.dataset.dto.DatasetDto.SearchReq;
import com.kamco.cd.training.dataset.dto.DatasetDto.SelectDataSet;
import com.kamco.cd.training.dataset.dto.DatasetDto.SelectTransferDataSet;
import com.kamco.cd.training.postgres.entity.DatasetEntity;
import com.kamco.cd.training.postgres.entity.QDatasetEntity;
import com.kamco.cd.training.postgres.entity.QDatasetObjEntity;
import com.kamco.cd.training.postgres.entity.QModelDatasetMappEntity;
import com.kamco.cd.training.postgres.entity.QModelMasterEntity;
import com.querydsl.core.BooleanBuilder;
import com.querydsl.core.types.Projections;
import com.querydsl.core.types.dsl.CaseBuilder;
import com.querydsl.core.types.dsl.Expressions;
import com.querydsl.core.types.dsl.NumberExpression;
import com.querydsl.jpa.impl.JPAQueryFactory;
import java.util.List;
import java.util.Optional;
@@ -29,7 +45,7 @@ public class DatasetRepositoryImpl implements DatasetRepositoryCustom {
* @return 페이징 처리된 데이터셋 Entity 목록
*/
@Override
public Page<DatasetEntity> findDatasetList(DatasetDto.SearchReq searchReq) {
public Page<DatasetEntity> findDatasetList(SearchReq searchReq) {
Pageable pageable = searchReq.toPageable();
BooleanBuilder builder = new BooleanBuilder();
@@ -40,8 +56,8 @@ public class DatasetRepositoryImpl implements DatasetRepositoryCustom {
}
// 구분
if (StringUtils.isNotBlank(searchReq.getGroupTitle())) {
builder.and(dataset.groupTitle.eq(searchReq.getGroupTitle()));
if (StringUtils.isNotBlank(searchReq.getDataType())) {
builder.and(dataset.dataType.eq(searchReq.getDataType()));
}
// Entity 직접 조회 (Projections 사용 지양)
@@ -57,7 +73,11 @@ public class DatasetRepositoryImpl implements DatasetRepositoryCustom {
// Count 쿼리 별도 실행 (null safe handling)
long total =
Optional.ofNullable(
queryFactory.select(dataset.count()).from(dataset).where(builder).fetchOne())
queryFactory
.select(dataset.count())
.from(dataset)
.where(builder.and(dataset.deleted.isFalse()))
.fetchOne())
.orElse(0L);
return new PageImpl<>(content, pageable, total);
@@ -72,4 +92,385 @@ public class DatasetRepositoryImpl implements DatasetRepositoryCustom {
.where(dataset.uuid.eq(id), dataset.deleted.isFalse())
.fetchOne());
}
@Override
public List<SelectDataSet> getDatasetSelectG1List(DatasetReq req) {
BooleanBuilder builder = new BooleanBuilder();
if (StringUtils.isNotBlank(req.getDataType()) && !"CURRENT".equals(req.getDataType())) {
builder.and(dataset.dataType.eq(req.getDataType()));
}
if (StringUtils.isNotBlank(req.getDataType()) && !"CURRENT".equals(req.getDataType())) {
builder.and(dataset.dataType.eq(req.getDataType()));
}
if (req.getIds() != null) {
builder.and(dataset.id.in(req.getIds()));
}
return queryFactory
.select(
Projections.constructor(
SelectDataSet.class,
Expressions.constant(req.getModelNo()),
dataset.id,
dataset.uuid,
dataset.dataType,
dataset.title,
dataset.roundNo,
dataset.compareYyyy,
dataset.targetYyyy,
dataset.memo,
new CaseBuilder()
.when(datasetObjEntity.targetClassCd.eq("building"))
.then(1)
.otherwise(0)
.sum(),
new CaseBuilder()
.when(datasetObjEntity.targetClassCd.eq("container"))
.then(1)
.otherwise(0)
.sum()))
.from(dataset)
.leftJoin(datasetObjEntity)
.on(dataset.id.eq(datasetObjEntity.datasetUid))
.where(builder) // datasetObjEntity.targetClassCd.in("building", "container").and(
.groupBy(
dataset.id,
dataset.uuid,
dataset.dataType,
dataset.title,
dataset.roundNo,
dataset.memo)
.orderBy(dataset.createdDttm.desc())
.fetch();
}
@Override
public List<SelectTransferDataSet> getDatasetTransferSelectG1List(Long modelId) {
QModelMasterEntity beforeMaster = new QModelMasterEntity("beforeMaster");
QModelDatasetMappEntity beforeMapp = new QModelDatasetMappEntity("beforeMapp");
QDatasetEntity beforeDataset = new QDatasetEntity("beforeDataset");
QDatasetObjEntity beforeObj = new QDatasetObjEntity("beforeObj");
return queryFactory
.select(
Projections.constructor(
SelectTransferDataSet.class,
// ===== 현재 =====
modelMasterEntity.modelNo,
dataset.id,
dataset.uuid,
dataset.dataType,
dataset.title,
dataset.roundNo,
dataset.compareYyyy,
dataset.targetYyyy,
dataset.memo,
new CaseBuilder()
.when(datasetObjEntity.targetClassCd.eq("building"))
.then(1)
.otherwise(0)
.sum(),
new CaseBuilder()
.when(datasetObjEntity.targetClassCd.eq("container"))
.then(1)
.otherwise(0)
.sum(),
// ===== before (join으로) =====
beforeMaster.modelNo,
beforeDataset.id,
beforeDataset.uuid,
beforeDataset.dataType,
beforeDataset.title,
beforeDataset.roundNo,
beforeDataset.compareYyyy,
beforeDataset.targetYyyy,
beforeDataset.memo,
new CaseBuilder()
.when(beforeObj.targetClassCd.eq("building"))
.then(1)
.otherwise(0)
.sum(),
new CaseBuilder()
.when(beforeObj.targetClassCd.eq("container"))
.then(1)
.otherwise(0)
.sum()))
.from(modelMasterEntity)
// ===== 현재 dataset join =====
.leftJoin(modelDatasetMappEntity)
.on(modelDatasetMappEntity.modelUid.eq(modelMasterEntity.id))
.leftJoin(dataset)
.on(modelDatasetMappEntity.datasetUid.eq(dataset.id))
.leftJoin(datasetObjEntity)
.on(dataset.id.eq(datasetObjEntity.datasetUid))
// ===== before 모델 join =====
.leftJoin(beforeMaster)
.on(beforeMaster.id.eq(modelMasterEntity.beforeModelId))
.leftJoin(beforeMapp)
.on(beforeMapp.modelUid.eq(beforeMaster.id))
.leftJoin(beforeDataset)
.on(beforeMapp.datasetUid.eq(beforeDataset.id))
.leftJoin(beforeObj)
.on(beforeDataset.id.eq(beforeObj.datasetUid))
.where(modelMasterEntity.id.eq(modelId))
.groupBy(
modelMasterEntity.modelNo,
dataset.id,
dataset.uuid,
dataset.dataType,
dataset.title,
dataset.roundNo,
dataset.compareYyyy,
dataset.targetYyyy,
dataset.memo,
beforeMaster.modelNo,
beforeDataset.id,
beforeDataset.uuid,
beforeDataset.dataType,
beforeDataset.title,
beforeDataset.roundNo,
beforeDataset.compareYyyy,
beforeDataset.targetYyyy,
beforeDataset.memo)
.orderBy(dataset.createdDttm.desc())
.fetch();
}
@Override
public List<SelectDataSet> getDatasetSelectG2G3List(DatasetReq req) {
BooleanBuilder builder = new BooleanBuilder();
NumberExpression<Long> selectedCnt = null;
NumberExpression<Long> wasteCnt =
datasetObjEntity.targetClassCd.when("waste").then(1L).otherwise(0L).sum();
NumberExpression<Long> elseCnt =
new CaseBuilder()
.when(datasetObjEntity.targetClassCd.notIn("building", "container", "waste"))
.then(1L)
.otherwise(0L)
.sum();
if (StringUtils.isNotBlank(req.getModelNo())) {
if (req.getModelNo().equals(ModelType.G2.getId())) {
selectedCnt = wasteCnt;
} else {
selectedCnt = elseCnt;
}
}
if (StringUtils.isNotBlank(req.getDataType())) {
if (!"CURRENT".equals(req.getDataType())) {
builder.and(dataset.dataType.eq(req.getDataType()));
}
}
if (req.getIds() != null) {
builder.and(dataset.id.in(req.getIds()));
}
return queryFactory
.select(
Projections.constructor(
SelectDataSet.class,
Expressions.constant(req.getModelNo()),
dataset.id,
dataset.uuid,
dataset.dataType,
dataset.title,
dataset.roundNo,
dataset.compareYyyy,
dataset.targetYyyy,
dataset.memo,
selectedCnt.as("cnt")))
.from(dataset)
.leftJoin(datasetObjEntity)
.on(dataset.id.eq(datasetObjEntity.datasetUid))
.where(builder)
.groupBy(
dataset.id,
dataset.uuid,
dataset.dataType,
dataset.title,
dataset.roundNo,
dataset.memo)
.orderBy(dataset.createdDttm.desc())
.fetch();
}
@Override
public List<SelectTransferDataSet> getDatasetTransferSelectG2G3List(
Long modelId, String modelNo) {
// before join용
QModelMasterEntity beforeMaster = new QModelMasterEntity("beforeMaster");
QModelDatasetMappEntity beforeMapp = new QModelDatasetMappEntity("beforeMapp");
QDatasetEntity beforeDataset = new QDatasetEntity("beforeDataset");
QDatasetObjEntity beforeObj = new QDatasetObjEntity("beforeObj");
BooleanBuilder builder = new BooleanBuilder();
NumberExpression<Long> wasteCnt =
datasetObjEntity.targetClassCd.when("waste").then(1L).otherwise(0L).sum();
NumberExpression<Long> elseCnt =
new CaseBuilder()
.when(datasetObjEntity.targetClassCd.notIn("building", "container", "waste"))
.then(1L)
.otherwise(0L)
.sum();
NumberExpression<Long> selectedCnt = ModelType.G2.getId().equals(modelNo) ? wasteCnt : elseCnt;
// before도 동일 로직으로 cnt 계산
NumberExpression<Long> beforeWasteCnt =
beforeObj.targetClassCd.when("waste").then(1L).otherwise(0L).sum();
NumberExpression<Long> beforeElseCnt =
new CaseBuilder()
.when(beforeObj.targetClassCd.notIn("building", "container", "waste"))
.then(1L)
.otherwise(0L)
.sum();
NumberExpression<Long> beforeSelectedCnt =
ModelType.G2.getId().equals(modelNo) ? beforeWasteCnt : beforeElseCnt;
return queryFactory
.select(
Projections.constructor(
SelectTransferDataSet.class,
// ===== 현재 =====
modelMasterEntity.modelNo, // modelNo 파라미터 사용 (req.getModelNo() 제거)
dataset.id,
dataset.uuid,
dataset.dataType,
dataset.title,
dataset.roundNo,
dataset.compareYyyy,
dataset.targetYyyy,
dataset.memo,
selectedCnt, // classCount 자리에 들어가는 cnt (Long)
// ===== before =====
beforeMaster.modelNo,
beforeDataset.id,
beforeDataset.uuid,
beforeDataset.dataType,
beforeDataset.title,
beforeDataset.roundNo,
beforeDataset.compareYyyy,
beforeDataset.targetYyyy,
beforeDataset.memo,
beforeSelectedCnt))
.from(modelMasterEntity)
// ===== 현재 dataset =====
.leftJoin(modelDatasetMappEntity)
.on(modelDatasetMappEntity.modelUid.eq(modelMasterEntity.id))
.leftJoin(dataset)
.on(modelDatasetMappEntity.datasetUid.eq(dataset.id))
.leftJoin(datasetObjEntity)
.on(dataset.id.eq(datasetObjEntity.datasetUid))
// ===== before dataset =====
.leftJoin(beforeMaster)
.on(beforeMaster.id.eq(modelMasterEntity.beforeModelId))
.leftJoin(beforeMapp)
.on(beforeMapp.modelUid.eq(beforeMaster.id))
.leftJoin(beforeDataset)
.on(beforeMapp.datasetUid.eq(beforeDataset.id))
.leftJoin(beforeObj)
.on(beforeDataset.id.eq(beforeObj.datasetUid))
.where(modelMasterEntity.id.eq(modelId).and(builder))
// sum() 때문에 groupBy 필요
.groupBy(
dataset.id,
dataset.uuid,
dataset.dataType,
dataset.title,
dataset.roundNo,
dataset.compareYyyy,
dataset.targetYyyy,
dataset.memo,
beforeMaster.modelNo,
beforeDataset.id,
beforeDataset.uuid,
beforeDataset.dataType,
beforeDataset.title,
beforeDataset.roundNo,
beforeDataset.compareYyyy,
beforeDataset.targetYyyy,
beforeDataset.memo)
.orderBy(dataset.createdDttm.desc())
.fetch();
}
@Override
public Long getDatasetMaxStage(int compareYyyy, int targetYyyy) {
return queryFactory
.select(dataset.roundNo.max().coalesce(0L))
.from(dataset)
.where(dataset.compareYyyy.eq(compareYyyy), dataset.targetYyyy.eq(targetYyyy))
.fetchOne();
}
@Override
public Long insertDatasetMngData(DatasetMngRegDto mngRegDto) {
queryFactory
.insert(dataset)
.columns(
dataset.uid,
dataset.dataType,
dataset.compareYyyy,
dataset.targetYyyy,
dataset.roundNo,
dataset.totalSize,
dataset.title,
dataset.memo,
dataset.datasetPath)
.values(
mngRegDto.getUid(),
mngRegDto.getDataType(),
mngRegDto.getCompareYyyy(),
mngRegDto.getTargetYyyy(),
mngRegDto.getRoundNo(),
mngRegDto.getTotalSize(),
mngRegDto.getTitle(),
mngRegDto.getMemo(),
mngRegDto.getDatasetPath())
.execute();
return queryFactory
.select(dataset.id)
.from(dataset)
.where(dataset.uid.eq(mngRegDto.getUid()), dataset.deleted.isFalse())
.fetchOne();
}
@Override
public List<String> findDatasetUid(List<Long> datasetIds) {
return queryFactory.select(dataset.uid).from(dataset).where(dataset.id.in(datasetIds)).fetch();
}
@Override
public Long findDatasetByUidExistsCnt(String uid) {
return queryFactory
.select(dataset.id.count())
.from(dataset)
.where(dataset.uid.eq(uid), dataset.deleted.isFalse())
.fetchOne();
}
}

View File

@@ -1,9 +1,12 @@
package com.kamco.cd.training.postgres.repository.hyperparam;
import com.kamco.cd.training.postgres.entity.ModelHyperParamEntity;
import java.util.Optional;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;
@Repository
public interface HyperParamRepository
extends JpaRepository<ModelHyperParamEntity, Long>, HyperParamRepositoryCustom {}
extends JpaRepository<ModelHyperParamEntity, Long>, HyperParamRepositoryCustom {
Optional<ModelHyperParamEntity> findByHyperVer(String hyperVer);
}

View File

@@ -1,7 +1,10 @@
package com.kamco.cd.training.postgres.repository.hyperparam;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto.SearchReq;
import com.kamco.cd.training.postgres.entity.ModelHyperParamEntity;
import java.util.List;
import java.util.Optional;
import java.util.UUID;
import org.springframework.data.domain.Page;
@@ -13,9 +16,41 @@ public interface HyperParamRepositoryCustom {
*
* @return
*/
@Deprecated
Optional<ModelHyperParamEntity> findHyperParamVer();
/**
* 모델 타입별 마지막 버전 조회
*
* @param modelType 모델 타입
* @return
*/
Optional<ModelHyperParamEntity> findHyperParamVerByModelType(ModelType modelType);
Optional<ModelHyperParamEntity> findHyperParamByHyperVer(String hyperVer);
/**
* 하이퍼 파라미터 상세조회
*
* @param uuid
* @return
*/
Optional<ModelHyperParamEntity> findHyperParamByUuid(UUID uuid);
Page<HyperParamDto.List> findByHyperVerList(HyperParamDto.SearchReq req);
/**
* 하이퍼 파라미터 목록 조회
*
* @param model
* @param req
* @return
*/
Page<HyperParamDto.List> findByHyperVerList(ModelType model, SearchReq req);
/**
* 하이퍼 파라미터 모델타입으로 조회
*
* @param modelType
* @return
*/
List<ModelHyperParamEntity> getHyperParamByType(ModelType modelType);
}

View File

@@ -2,12 +2,13 @@ package com.kamco.cd.training.postgres.repository.hyperparam;
import static com.kamco.cd.training.postgres.entity.QModelHyperParamEntity.modelHyperParamEntity;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto.HyperType;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto.SearchReq;
import com.kamco.cd.training.postgres.entity.ModelHyperParamEntity;
import com.querydsl.core.BooleanBuilder;
import com.querydsl.core.types.Projections;
import com.querydsl.core.types.dsl.NumberExpression;
import com.querydsl.jpa.impl.JPAQuery;
import com.querydsl.jpa.impl.JPAQueryFactory;
import java.time.ZoneId;
@@ -35,28 +36,66 @@ public class HyperParamRepositoryImpl implements HyperParamRepositoryCustom {
queryFactory
.select(modelHyperParamEntity)
.from(modelHyperParamEntity)
.where(modelHyperParamEntity.delYn.eq("N"))
.where(modelHyperParamEntity.delYn.isFalse())
.orderBy(modelHyperParamEntity.hyperVer.desc())
.limit(1)
.fetchOne());
}
@Override
public Optional<ModelHyperParamEntity> findHyperParamVerByModelType(ModelType modelType) {
return Optional.ofNullable(
queryFactory
.select(modelHyperParamEntity)
.from(modelHyperParamEntity)
.where(
modelHyperParamEntity
.delYn
.isFalse()
.and(modelHyperParamEntity.modelType.eq(modelType)))
.orderBy(modelHyperParamEntity.hyperVer.desc())
.limit(1)
.fetchOne());
}
@Override
public Optional<ModelHyperParamEntity> findHyperParamByHyperVer(String hyperVer) {
return Optional.ofNullable(
queryFactory
.select(modelHyperParamEntity)
.from(modelHyperParamEntity)
.where(
modelHyperParamEntity
.delYn
.isFalse()
.and(modelHyperParamEntity.hyperVer.eq(hyperVer)))
.limit(1)
.fetchOne());
}
@Override
public Optional<ModelHyperParamEntity> findHyperParamByUuid(UUID uuid) {
return Optional.ofNullable(
queryFactory
.select(modelHyperParamEntity)
.from(modelHyperParamEntity)
.where(modelHyperParamEntity.delYn.eq("N").and(modelHyperParamEntity.uuid.eq(uuid)))
.where(modelHyperParamEntity.uuid.eq(uuid))
.fetchOne());
}
@Override
public Page<HyperParamDto.List> findByHyperVerList(HyperParamDto.SearchReq req) {
public Page<HyperParamDto.List> findByHyperVerList(ModelType model, SearchReq req) {
Pageable pageable = req.toPageable();
BooleanBuilder builder = new BooleanBuilder();
builder.and(modelHyperParamEntity.delYn.eq("N"));
builder.and(modelHyperParamEntity.delYn.isFalse());
if (model != null) {
builder.and(modelHyperParamEntity.modelType.eq(model));
}
if (req.getHyperVer() != null && !req.getHyperVer().isEmpty()) {
// 버전
@@ -80,26 +119,18 @@ public class HyperParamRepositoryImpl implements HyperParamRepositoryCustom {
}
}
NumberExpression<Long> totalUseCnt =
modelHyperParamEntity
.m1UseCnt
.coalesce(0L)
.add(modelHyperParamEntity.m2UseCnt.coalesce(0L))
.add(modelHyperParamEntity.m3UseCnt.coalesce(0L));
JPAQuery<HyperParamDto.List> query =
queryFactory
.select(
Projections.constructor(
HyperParamDto.List.class,
modelHyperParamEntity.uuid,
modelHyperParamEntity.modelType.as("model"),
modelHyperParamEntity.hyperVer,
modelHyperParamEntity.createdDttm,
modelHyperParamEntity.lastUsedDttm,
modelHyperParamEntity.m1UseCnt,
modelHyperParamEntity.m2UseCnt,
modelHyperParamEntity.m3UseCnt,
totalUseCnt.as("totalUseCnt")))
modelHyperParamEntity.memo,
modelHyperParamEntity.totalUseCnt))
.from(modelHyperParamEntity)
.where(builder);
@@ -124,8 +155,11 @@ public class HyperParamRepositoryImpl implements HyperParamRepositoryCustom {
asc
? modelHyperParamEntity.lastUsedDttm.asc()
: modelHyperParamEntity.lastUsedDttm.desc());
case "totalUseCnt" -> query.orderBy(asc ? totalUseCnt.asc() : totalUseCnt.desc());
case "totalUseCnt" ->
query.orderBy(
asc
? modelHyperParamEntity.totalUseCnt.asc()
: modelHyperParamEntity.totalUseCnt.desc());
default -> query.orderBy(modelHyperParamEntity.createdDttm.desc());
}
@@ -145,4 +179,17 @@ public class HyperParamRepositoryImpl implements HyperParamRepositoryCustom {
return new PageImpl<>(content, pageable, totalCount);
}
@Override
public List<ModelHyperParamEntity> getHyperParamByType(ModelType modelType) {
return queryFactory
.select(modelHyperParamEntity)
.from(modelHyperParamEntity)
.where(
modelHyperParamEntity
.delYn
.isFalse()
.and(modelHyperParamEntity.modelType.eq(modelType)))
.fetch();
}
}

View File

@@ -1,6 +1,7 @@
package com.kamco.cd.training.postgres.repository.log;
import com.kamco.cd.training.log.dto.AuditLogDto;
import com.kamco.cd.training.log.dto.AuditLogDto.DownloadReq;
import java.time.LocalDate;
import org.springframework.data.domain.Page;
@@ -15,6 +16,9 @@ public interface AuditLogRepositoryCustom {
Page<AuditLogDto.UserAuditList> findLogByAccount(
AuditLogDto.searchReq searchReq, String searchValue);
Page<AuditLogDto.DownloadRes> findDownloadLog(
AuditLogDto.searchReq searchReq, DownloadReq downloadReq);
Page<AuditLogDto.DailyDetail> findLogByDailyResult(
AuditLogDto.searchReq searchReq, LocalDate logDate);

View File

@@ -6,32 +6,42 @@ import static com.kamco.cd.training.postgres.entity.QMemberEntity.memberEntity;
import static com.kamco.cd.training.postgres.entity.QMenuEntity.menuEntity;
import com.kamco.cd.training.log.dto.AuditLogDto;
import com.kamco.cd.training.log.dto.AuditLogDto.DownloadReq;
import com.kamco.cd.training.log.dto.AuditLogDto.searchReq;
import com.kamco.cd.training.log.dto.ErrorLogDto;
import com.kamco.cd.training.log.dto.EventStatus;
import com.kamco.cd.training.log.dto.EventType;
import com.kamco.cd.training.postgres.entity.AuditLogEntity;
import com.kamco.cd.training.postgres.entity.QMenuEntity;
import com.querydsl.core.BooleanBuilder;
import com.querydsl.core.types.Projections;
import com.querydsl.core.types.dsl.*;
import com.querydsl.jpa.impl.JPAQueryFactory;
import io.micrometer.common.util.StringUtils;
import java.time.LocalDate;
import java.time.LocalDateTime;
import java.time.ZoneId;
import java.time.ZonedDateTime;
import java.util.List;
import java.util.Objects;
import lombok.RequiredArgsConstructor;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.PageImpl;
import org.springframework.data.domain.Pageable;
import org.springframework.data.jpa.repository.support.QuerydslRepositorySupport;
import org.springframework.stereotype.Repository;
@Repository
@RequiredArgsConstructor
public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
public class AuditLogRepositoryImpl extends QuerydslRepositorySupport
implements AuditLogRepositoryCustom {
private static final ZoneId ZONE = ZoneId.of("Asia/Seoul");
private final JPAQueryFactory queryFactory;
private final StringExpression NULL_STRING = Expressions.stringTemplate("cast(null as text)");
public AuditLogRepositoryImpl(JPAQueryFactory queryFactory) {
super(AuditLogEntity.class);
this.queryFactory = queryFactory;
}
@Override
public Page<AuditLogDto.DailyAuditList> findLogByDaily(
AuditLogDto.searchReq searchReq, LocalDate startDate, LocalDate endDate) {
@@ -87,7 +97,7 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
.from(auditLogEntity)
.leftJoin(menuEntity)
.on(auditLogEntity.menuUid.eq(menuEntity.menuUid))
.where(menuNameEquals(searchValue))
.where(auditLogEntity.menuUid.ne("SYSTEM"), menuNameEquals(searchValue))
.groupBy(auditLogEntity.menuUid)
.offset(pageable.getOffset())
.limit(pageable.getPageSize())
@@ -128,7 +138,7 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
.from(auditLogEntity)
.leftJoin(memberEntity)
.on(auditLogEntity.userUid.eq(memberEntity.id))
.where(loginIdOrUsernameContains(searchValue))
.where(auditLogEntity.userUid.isNotNull(), loginIdOrUsernameContains(searchValue))
.groupBy(auditLogEntity.userUid, memberEntity.employeeNo, memberEntity.name)
.offset(pageable.getOffset())
.limit(pageable.getPageSize())
@@ -147,6 +157,62 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
return new PageImpl<>(foundContent, pageable, countQuery);
}
@Override
public Page<AuditLogDto.DownloadRes> findDownloadLog(
AuditLogDto.searchReq searchReq, DownloadReq req) {
Pageable pageable = searchReq.toPageable();
BooleanBuilder whereBuilder = new BooleanBuilder();
whereBuilder.and(auditLogEntity.eventStatus.ne(EventStatus.valueOf("FAILED")));
whereBuilder.and(auditLogEntity.eventType.eq(EventType.valueOf("DOWNLOAD")));
// if (req.getMenuId() != null && !req.getMenuId().isEmpty()) {
// whereBuilder.and(auditLogEntity.menuUid.eq(req.getMenuId()));
// }
if (req.getUuid() != null) {
whereBuilder.and(auditLogEntity.requestUri.contains(req.getRequestUri()));
whereBuilder.and(auditLogEntity.downloadUuid.eq(req.getUuid()));
}
if (req.getSearchValue() != null && !req.getSearchValue().isEmpty()) {
whereBuilder.and(
memberEntity
.name
.contains(req.getSearchValue())
.or(memberEntity.employeeNo.contains(req.getSearchValue())));
}
List<AuditLogDto.DownloadRes> foundContent =
queryFactory
.select(
Projections.constructor(
AuditLogDto.DownloadRes.class,
memberEntity.name,
memberEntity.employeeNo,
auditLogEntity.createdDate.as("downloadDttm")))
.from(auditLogEntity)
.leftJoin(memberEntity)
.on(auditLogEntity.userUid.eq(memberEntity.id))
.where(whereBuilder, createdDateBetween(req.getStartDate(), req.getEndDate()))
.offset(pageable.getOffset())
.limit(pageable.getPageSize())
.orderBy(auditLogEntity.createdDate.desc())
.fetch();
Long countQuery =
queryFactory
.select(auditLogEntity.userUid.countDistinct())
.from(auditLogEntity)
.leftJoin(memberEntity)
.on(auditLogEntity.userUid.eq(memberEntity.id))
.where(whereBuilder, createdDateBetween(req.getStartDate(), req.getEndDate()))
.fetchOne();
return new PageImpl<>(foundContent, pageable, countQuery);
}
@Override
public Page<AuditLogDto.DailyDetail> findLogByDailyResult(
AuditLogDto.searchReq searchReq, LocalDate logDate) {
@@ -176,6 +242,9 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
memberEntity.employeeNo.as("loginId"),
menuEntity.menuNm.as("menuName"),
auditLogEntity.eventType.as("eventType"),
Expressions.stringTemplate(
"to_char({0}, 'YYYY-MM-DD HH24:MI')", auditLogEntity.createdDate)
.as("logDateTime"),
Projections.constructor(
AuditLogDto.LogDetail.class,
Expressions.constant("한국자산관리공사"), // serviceName
@@ -184,7 +253,7 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
menuEntity.menuUrl.as("menuUrl"),
menuEntity.description.as("menuDescription"),
menuEntity.menuOrder.as("sortOrder"),
menuEntity.isUse.as("used"))))
menuEntity.isUse.as("used")))) // TODO
.from(auditLogEntity)
.leftJoin(menuEntity)
.on(auditLogEntity.menuUid.eq(menuEntity.menuUid))
@@ -238,8 +307,8 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
AuditLogDto.MenuDetail.class,
auditLogEntity.id.as("logId"),
Expressions.stringTemplate(
"to_char({0}, 'YYYY-MM-DD')", auditLogEntity.createdDate)
.as("logDateTime"), // ??
"to_char({0}, 'YYYY-MM-DD HH24:MI')", auditLogEntity.createdDate)
.as("logDateTime"),
memberEntity.name.as("userName"),
memberEntity.employeeNo.as("loginId"),
auditLogEntity.eventType.as("eventType"),
@@ -305,7 +374,7 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
AuditLogDto.UserDetail.class,
auditLogEntity.id.as("logId"),
Expressions.stringTemplate(
"to_char({0}, 'YYYY-MM-DD')", auditLogEntity.createdDate)
"to_char({0}, 'YYYY-MM-DD HH24:MI')", auditLogEntity.createdDate)
.as("logDateTime"),
menuEntity.menuNm.as("menuName"),
auditLogEntity.eventType.as("eventType"),
@@ -349,12 +418,23 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
if (Objects.isNull(startDate) || Objects.isNull(endDate)) {
return null;
}
LocalDateTime startDateTime = startDate.atStartOfDay();
LocalDateTime endDateTime = endDate.plusDays(1).atStartOfDay();
ZoneId zoneId = ZoneId.of("Asia/Seoul");
ZonedDateTime startDateTime = startDate.atStartOfDay(zoneId);
ZonedDateTime endDateTime = endDate.plusDays(1).atStartOfDay(zoneId);
return auditLogEntity
.createdDate
.goe(ZonedDateTime.from(startDateTime))
.and(auditLogEntity.createdDate.lt(ZonedDateTime.from(endDateTime)));
.goe(startDateTime)
.and(auditLogEntity.createdDate.lt(endDateTime));
}
private BooleanExpression createdDateBetween(LocalDate startDate, LocalDate endDate) {
if (startDate == null || endDate == null) {
return null;
}
ZonedDateTime start = startDate.atStartOfDay(ZONE);
ZonedDateTime endExclusive = endDate.plusDays(1).atStartOfDay(ZONE);
return auditLogEntity.createdDate.goe(start).and(auditLogEntity.createdDate.lt(endExclusive));
}
private BooleanExpression menuNameEquals(String searchValue) {
@@ -393,11 +473,11 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
}
private BooleanExpression eventEndedAtEqDate(LocalDate logDate) {
StringExpression eventEndedDate =
Expressions.stringTemplate("to_char({0}, 'YYYY-MM-DD')", auditLogEntity.createdDate);
LocalDateTime comparisonDate = logDate.atStartOfDay();
ZoneId zoneId = ZoneId.of("Asia/Seoul");
ZonedDateTime start = logDate.atStartOfDay(zoneId);
ZonedDateTime end = logDate.plusDays(1).atStartOfDay(zoneId);
return eventEndedDate.eq(comparisonDate.toString());
return auditLogEntity.createdDate.goe(start).and(auditLogEntity.createdDate.lt(end));
}
private BooleanExpression menuUidEq(String menuUid) {
@@ -410,7 +490,7 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
private NumberExpression<Integer> readCount() {
return new CaseBuilder()
.when(auditLogEntity.eventType.eq(EventType.READ))
.when(auditLogEntity.eventType.in(EventType.LIST, EventType.DETAIL))
.then(1)
.otherwise(0)
.sum();
@@ -418,7 +498,7 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
private NumberExpression<Integer> cudCount() {
return new CaseBuilder()
.when(auditLogEntity.eventType.in(EventType.CREATE, EventType.UPDATE, EventType.DELETE))
.when(auditLogEntity.eventType.in(EventType.ADDED, EventType.MODIFIED, EventType.REMOVE))
.then(1)
.otherwise(0)
.sum();
@@ -426,7 +506,7 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
private NumberExpression<Integer> printCount() {
return new CaseBuilder()
.when(auditLogEntity.eventType.eq(EventType.PRINT))
.when(auditLogEntity.eventType.eq(EventType.OTHER))
.then(1)
.otherwise(0)
.sum();

View File

@@ -8,29 +8,35 @@ import static com.kamco.cd.training.postgres.entity.QMenuEntity.menuEntity;
import com.kamco.cd.training.log.dto.ErrorLogDto;
import com.kamco.cd.training.log.dto.EventStatus;
import com.kamco.cd.training.log.dto.EventType;
import com.kamco.cd.training.postgres.entity.AuditLogEntity;
import com.querydsl.core.types.Projections;
import com.querydsl.core.types.dsl.BooleanExpression;
import com.querydsl.core.types.dsl.Expressions;
import com.querydsl.core.types.dsl.StringExpression;
import com.querydsl.jpa.impl.JPAQueryFactory;
import java.time.LocalDate;
import java.time.LocalDateTime;
import java.time.ZoneId;
import java.time.ZonedDateTime;
import java.util.List;
import java.util.Objects;
import lombok.RequiredArgsConstructor;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.PageImpl;
import org.springframework.data.domain.Pageable;
import org.springframework.data.jpa.repository.support.QuerydslRepositorySupport;
import org.springframework.stereotype.Repository;
@Repository
@RequiredArgsConstructor
public class ErrorLogRepositoryImpl implements ErrorLogRepositoryCustom {
public class ErrorLogRepositoryImpl extends QuerydslRepositorySupport
implements ErrorLogRepositoryCustom {
private final JPAQueryFactory queryFactory;
private final StringExpression NULL_STRING = Expressions.stringTemplate("cast(null as text)");
public ErrorLogRepositoryImpl(JPAQueryFactory queryFactory) {
super(AuditLogEntity.class);
this.queryFactory = queryFactory;
}
@Override
public Page<ErrorLogDto.Basic> findLogByError(ErrorLogDto.ErrorSearchReq searchReq) {
Pageable pageable = searchReq.toPageable();
@@ -52,7 +58,7 @@ public class ErrorLogRepositoryImpl implements ErrorLogRepositoryCustom {
errorLogEntity.errorMessage.as("errorMessage"),
errorLogEntity.stackTrace.as("errorDetail"),
Expressions.stringTemplate(
"to_char({0}, 'YYYY-MM-DD')", errorLogEntity.createdDate)))
"to_char({0}, 'YYYY-MM-DD HH24:MI:SS.FF3')", errorLogEntity.createdDate)))
.from(errorLogEntity)
.leftJoin(auditLogEntity)
.on(errorLogEntity.id.eq(auditLogEntity.errorLogUid))
@@ -94,12 +100,14 @@ public class ErrorLogRepositoryImpl implements ErrorLogRepositoryCustom {
if (Objects.isNull(startDate) || Objects.isNull(endDate)) {
return null;
}
LocalDateTime startDateTime = startDate.atStartOfDay();
LocalDateTime endDateTime = endDate.plusDays(1).atStartOfDay();
ZoneId zoneId = ZoneId.of("Asia/Seoul");
ZonedDateTime startDateTime = startDate.atStartOfDay(zoneId);
ZonedDateTime endDateTime = endDate.plusDays(1).atStartOfDay(zoneId);
return auditLogEntity
.createdDate
.goe(ZonedDateTime.from(startDateTime))
.and(auditLogEntity.createdDate.lt(ZonedDateTime.from(endDateTime)));
.goe(startDateTime)
.and(auditLogEntity.createdDate.lt(endDateTime));
}
private BooleanExpression eventStatusEqFailed() {

View File

@@ -0,0 +1,7 @@
package com.kamco.cd.training.postgres.repository.model;
import com.kamco.cd.training.postgres.entity.ModelConfigEntity;
import org.springframework.data.jpa.repository.JpaRepository;
public interface ModelConfigRepository
extends JpaRepository<ModelConfigEntity, Long>, ModelConfigRepositoryCustom {}

View File

@@ -0,0 +1,10 @@
package com.kamco.cd.training.postgres.repository.model;
import com.kamco.cd.training.model.dto.ModelConfigDto;
import java.util.Optional;
public interface ModelConfigRepositoryCustom {
Optional<ModelConfigDto.Basic> findModelConfigByModelId(Long modelId);
Optional<ModelConfigDto.TransferBasic> findModelTransferConfigByModelId(Long modelId);
}

View File

@@ -0,0 +1,81 @@
package com.kamco.cd.training.postgres.repository.model;
import static com.kamco.cd.training.postgres.entity.QModelConfigEntity.modelConfigEntity;
import static com.kamco.cd.training.postgres.entity.QModelMasterEntity.modelMasterEntity;
import com.kamco.cd.training.model.dto.ModelConfigDto.Basic;
import com.kamco.cd.training.model.dto.ModelConfigDto.TransferBasic;
import com.kamco.cd.training.postgres.entity.QModelConfigEntity;
import com.kamco.cd.training.postgres.entity.QModelMasterEntity;
import com.querydsl.core.types.Projections;
import com.querydsl.jpa.impl.JPAQueryFactory;
import java.util.Optional;
import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Repository;
@Repository
@RequiredArgsConstructor
public class ModelConfigRepositoryImpl implements ModelConfigRepositoryCustom {
private final JPAQueryFactory queryFactory;
@Override
public Optional<Basic> findModelConfigByModelId(Long modelId) {
return Optional.ofNullable(
queryFactory
.select(
Projections.constructor(
Basic.class,
modelConfigEntity.id,
modelConfigEntity.model.id,
modelConfigEntity.epochCount,
modelConfigEntity.trainPercent,
modelConfigEntity.validationPercent,
modelConfigEntity.testPercent,
modelConfigEntity.memo))
.from(modelConfigEntity)
.where(modelConfigEntity.model.id.eq(modelId))
.fetchOne());
}
@Override
public Optional<TransferBasic> findModelTransferConfigByModelId(Long modelId) {
QModelConfigEntity beforeConfig = new QModelConfigEntity("beforeConfig");
QModelMasterEntity beforeMaster = new QModelMasterEntity("beforeMaster");
return Optional.ofNullable(
queryFactory
.select(
Projections.constructor(
TransferBasic.class,
// ===== 현재 =====
modelConfigEntity.id,
modelConfigEntity.model.id,
modelConfigEntity.epochCount,
modelConfigEntity.trainPercent,
modelConfigEntity.validationPercent,
modelConfigEntity.testPercent,
modelConfigEntity.memo,
// ===== before =====
beforeConfig.id,
beforeConfig.model.id,
beforeConfig.epochCount,
beforeConfig.trainPercent,
beforeConfig.validationPercent,
beforeConfig.testPercent,
beforeConfig.memo))
.from(modelConfigEntity)
.innerJoin(modelConfigEntity.model, modelMasterEntity)
// before 모델 조인
.leftJoin(beforeMaster)
.on(beforeMaster.id.eq(modelMasterEntity.beforeModelId))
.leftJoin(beforeConfig)
.on(beforeConfig.model.id.eq(beforeMaster.id))
.where(modelMasterEntity.id.eq(modelId))
.fetchOne());
}
}

View File

@@ -6,4 +6,5 @@ import org.springframework.stereotype.Repository;
@Repository
public interface ModelDatasetMappRepository
extends JpaRepository<ModelDatasetMappEntity, ModelDatasetMappEntity.ModelDatasetMappId> {}
extends JpaRepository<ModelDatasetMappEntity, ModelDatasetMappEntity.ModelDatasetMappId>,
ModelDatasetMappRepositoryCustom {}

View File

@@ -0,0 +1,15 @@
package com.kamco.cd.training.postgres.repository.model;
import com.kamco.cd.training.postgres.entity.ModelDatasetMappEntity;
import com.kamco.cd.training.train.dto.ModelTrainLinkDto;
import java.util.List;
public interface ModelDatasetMappRepositoryCustom {
List<ModelDatasetMappEntity> findByModelUid(Long modelId);
List<ModelTrainLinkDto> findDatasetTrainPath(Long modelId);
List<ModelTrainLinkDto> findDatasetValPath(Long modelId);
List<ModelTrainLinkDto> findDatasetTestPath(Long modelId);
}

View File

@@ -0,0 +1,165 @@
package com.kamco.cd.training.postgres.repository.model;
import static com.kamco.cd.training.postgres.entity.QDatasetEntity.datasetEntity;
import static com.kamco.cd.training.postgres.entity.QModelDatasetMappEntity.modelDatasetMappEntity;
import static com.kamco.cd.training.postgres.entity.QModelMasterEntity.modelMasterEntity;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.postgres.entity.ModelDatasetMappEntity;
import com.kamco.cd.training.postgres.entity.QDatasetObjEntity;
import com.kamco.cd.training.postgres.entity.QDatasetTestObjEntity;
import com.kamco.cd.training.postgres.entity.QDatasetValObjEntity;
import com.kamco.cd.training.train.dto.ModelTrainLinkDto;
import com.querydsl.core.types.Projections;
import com.querydsl.jpa.impl.JPAQueryFactory;
import java.util.List;
import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Repository;
@Repository
@RequiredArgsConstructor
public class ModelDatasetMappRepositoryImpl implements ModelDatasetMappRepositoryCustom {
private final JPAQueryFactory queryFactory;
@Override
public List<ModelDatasetMappEntity> findByModelUid(Long modelId) {
return queryFactory
.select(modelDatasetMappEntity)
.from(modelDatasetMappEntity)
.where(modelDatasetMappEntity.modelUid.eq(modelId))
.fetch();
}
@Override
public List<ModelTrainLinkDto> findDatasetTrainPath(Long modelId) {
QDatasetObjEntity datasetObjEntity = QDatasetObjEntity.datasetObjEntity;
return queryFactory
.select(
Projections.constructor(
ModelTrainLinkDto.class,
modelMasterEntity.id,
modelMasterEntity.trainType,
modelMasterEntity.modelNo,
modelDatasetMappEntity.datasetUid,
datasetObjEntity.targetClassCd,
datasetObjEntity.comparePath,
datasetObjEntity.targetPath,
datasetObjEntity.labelPath,
datasetObjEntity.geojsonPath,
datasetEntity.uid))
.from(modelMasterEntity)
.leftJoin(modelDatasetMappEntity)
.on(modelDatasetMappEntity.modelUid.eq(modelMasterEntity.id))
.leftJoin(datasetEntity)
.on(datasetEntity.id.eq(modelDatasetMappEntity.datasetUid))
.leftJoin(datasetObjEntity)
.on(
datasetObjEntity
.datasetUid
.eq(modelDatasetMappEntity.datasetUid)
.and(
modelMasterEntity
.modelNo
.eq(ModelType.G1.getId())
.and(datasetObjEntity.targetClassCd.upper().in("CONTAINER", "BUILDING"))
.or(
modelMasterEntity
.modelNo
.eq(ModelType.G2.getId())
.and(datasetObjEntity.targetClassCd.upper().eq("WASTE")))
.or(modelMasterEntity.modelNo.eq(ModelType.G3.getId()))))
.where(modelMasterEntity.id.eq(modelId))
.fetch();
}
@Override
public List<ModelTrainLinkDto> findDatasetValPath(Long modelId) {
QDatasetValObjEntity datasetValObjEntity = QDatasetValObjEntity.datasetValObjEntity;
return queryFactory
.select(
Projections.constructor(
ModelTrainLinkDto.class,
modelMasterEntity.id,
modelMasterEntity.trainType,
modelMasterEntity.modelNo,
modelDatasetMappEntity.datasetUid,
datasetValObjEntity.targetClassCd,
datasetValObjEntity.comparePath,
datasetValObjEntity.targetPath,
datasetValObjEntity.labelPath,
datasetValObjEntity.geojsonPath,
datasetEntity.uid))
.from(modelMasterEntity)
.leftJoin(modelDatasetMappEntity)
.on(modelDatasetMappEntity.modelUid.eq(modelMasterEntity.id))
.leftJoin(datasetEntity)
.on(datasetEntity.id.eq(modelDatasetMappEntity.datasetUid))
.leftJoin(datasetValObjEntity)
.on(
datasetValObjEntity
.datasetUid
.eq(modelDatasetMappEntity.datasetUid)
.and(
modelMasterEntity
.modelNo
.eq(ModelType.G1.getId())
.and(datasetValObjEntity.targetClassCd.upper().in("CONTAINER", "BUILDING"))
.or(
modelMasterEntity
.modelNo
.eq(ModelType.G2.getId())
.and(datasetValObjEntity.targetClassCd.upper().eq("WASTE")))
.or(modelMasterEntity.modelNo.eq(ModelType.G3.getId()))))
.where(modelMasterEntity.id.eq(modelId))
.fetch();
}
@Override
public List<ModelTrainLinkDto> findDatasetTestPath(Long modelId) {
QDatasetTestObjEntity datasetTestObjEntity = QDatasetTestObjEntity.datasetTestObjEntity;
return queryFactory
.select(
Projections.constructor(
ModelTrainLinkDto.class,
modelMasterEntity.id,
modelMasterEntity.trainType,
modelMasterEntity.modelNo,
modelDatasetMappEntity.datasetUid,
datasetTestObjEntity.targetClassCd,
datasetTestObjEntity.comparePath,
datasetTestObjEntity.targetPath,
datasetTestObjEntity.labelPath,
datasetTestObjEntity.geojsonPath,
datasetEntity.uid))
.from(modelMasterEntity)
.leftJoin(modelDatasetMappEntity)
.on(modelDatasetMappEntity.modelUid.eq(modelMasterEntity.id))
.leftJoin(datasetEntity)
.on(datasetEntity.id.eq(modelDatasetMappEntity.datasetUid))
.leftJoin(datasetTestObjEntity)
.on(
datasetTestObjEntity
.datasetUid
.eq(modelDatasetMappEntity.datasetUid)
.and(
modelMasterEntity
.modelNo
.eq(ModelType.G1.getId())
.and(datasetTestObjEntity.targetClassCd.upper().in("CONTAINER", "BUILDING"))
.or(
modelMasterEntity
.modelNo
.eq(ModelType.G2.getId())
.and(datasetTestObjEntity.targetClassCd.upper().eq("WASTE")))
.or(modelMasterEntity.modelNo.eq(ModelType.G3.getId()))))
.where(modelMasterEntity.id.eq(modelId))
.fetch();
}
}

View File

@@ -0,0 +1,7 @@
package com.kamco.cd.training.postgres.repository.model;
import com.kamco.cd.training.postgres.entity.ModelDatasetEntity;
import org.springframework.data.jpa.repository.JpaRepository;
public interface ModelDatasetRepository
extends JpaRepository<ModelDatasetEntity, Long>, ModelDatasetRepositoryCustom {}

View File

@@ -0,0 +1,3 @@
package com.kamco.cd.training.postgres.repository.model;
public interface ModelDatasetRepositoryCustom {}

View File

@@ -0,0 +1,7 @@
package com.kamco.cd.training.postgres.repository.model;
import com.kamco.cd.training.postgres.entity.ModelMasterEntity;
import org.springframework.data.jpa.repository.JpaRepository;
public interface ModelDetailRepository
extends JpaRepository<ModelMasterEntity, Long>, ModelDetailRepositoryCustom {}

View File

@@ -0,0 +1,45 @@
package com.kamco.cd.training.postgres.repository.model;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.DetailSummary;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.HyperSummary;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.MappingDataset;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelBestEpoch;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelFileInfo;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelTestMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelTrainMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelValidationMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.TransferHyperSummary;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.ModelProgressStepDto;
import com.kamco.cd.training.postgres.entity.ModelMasterEntity;
import java.util.List;
import java.util.Optional;
import java.util.UUID;
public interface ModelDetailRepositoryCustom {
Optional<ModelMasterEntity> findByUuid(UUID uuid);
DetailSummary getModelDetailSummary(UUID uuid);
HyperSummary getByModelHyperParamSummary(UUID uuid);
TransferHyperSummary getByModelTransferHyperParamSummary(UUID uuid);
List<MappingDataset> getByModelMappingDataset(UUID uuid);
ModelMasterEntity findByModelByUUID(UUID uuid);
List<ModelTrainMetrics> getModelTrainMetricResult(UUID uuid);
List<ModelValidationMetrics> getModelValidationMetricResult(UUID uuid);
List<ModelTestMetrics> getModelTestMetricResult(UUID uuid);
ModelBestEpoch getModelTrainBestEpoch(UUID uuid);
ModelFileInfo getModelTrainFileInfo(UUID uuid);
List<ModelProgressStepDto> findModelTrainProgressInfo(UUID uuid);
ModelMasterEntity findByModelBeforeId(Long beforeModelId);
}

View File

@@ -0,0 +1,375 @@
package com.kamco.cd.training.postgres.repository.model;
import static com.kamco.cd.training.postgres.entity.QDatasetEntity.datasetEntity;
import static com.kamco.cd.training.postgres.entity.QModelDatasetEntity.modelDatasetEntity;
import static com.kamco.cd.training.postgres.entity.QModelDatasetMappEntity.modelDatasetMappEntity;
import static com.kamco.cd.training.postgres.entity.QModelHyperParamEntity.modelHyperParamEntity;
import static com.kamco.cd.training.postgres.entity.QModelMasterEntity.modelMasterEntity;
import static com.kamco.cd.training.postgres.entity.QModelMetricsTestEntity.modelMetricsTestEntity;
import static com.kamco.cd.training.postgres.entity.QModelMetricsTrainEntity.modelMetricsTrainEntity;
import static com.kamco.cd.training.postgres.entity.QModelMetricsValidationEntity.modelMetricsValidationEntity;
import com.kamco.cd.training.common.enums.TrainStatusType;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.DetailSummary;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.HyperSummary;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.MappingDataset;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelBestEpoch;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelFileInfo;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelTestMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelTrainMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelValidationMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.TransferHyperSummary;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.ModelProgressStepDto;
import com.kamco.cd.training.postgres.entity.ModelMasterEntity;
import com.kamco.cd.training.postgres.entity.QModelHyperParamEntity;
import com.kamco.cd.training.postgres.entity.QModelMasterEntity;
import com.querydsl.core.types.Expression;
import com.querydsl.core.types.Projections;
import com.querydsl.jpa.JPAExpressions;
import com.querydsl.jpa.impl.JPAQueryFactory;
import java.util.ArrayList;
import java.util.List;
import java.util.Optional;
import java.util.UUID;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Repository;
@Slf4j
@Repository
@RequiredArgsConstructor
public class ModelDetailRepositoryImpl implements ModelDetailRepositoryCustom {
private final JPAQueryFactory queryFactory;
/**
* 모델 조회
*
* @param uuid
* @return
*/
@Override
public Optional<ModelMasterEntity> findByUuid(UUID uuid) {
return Optional.ofNullable(
queryFactory
.select(modelMasterEntity)
.from(modelMasterEntity)
.where(modelMasterEntity.uuid.eq(uuid))
.fetchOne());
}
@Override
public DetailSummary getModelDetailSummary(UUID uuid) {
QModelMasterEntity beforeModel = new QModelMasterEntity("beforeModel"); // alias
Expression<UUID> beforeModelUuid =
com.querydsl.jpa.JPAExpressions.select(beforeModel.uuid)
.from(beforeModel)
.where(beforeModel.id.eq(modelMasterEntity.beforeModelId));
return queryFactory
.select(
Projections.constructor(
DetailSummary.class,
modelMasterEntity.id,
modelMasterEntity.uuid,
modelMasterEntity.modelNo,
modelMasterEntity.modelVer,
modelMasterEntity.step1StrtDttm,
modelMasterEntity.step2EndDttm,
modelMasterEntity.statusCd,
modelMasterEntity.trainType,
beforeModelUuid))
.from(modelMasterEntity)
.where(modelMasterEntity.uuid.eq(uuid))
.fetchOne();
}
@Override
public HyperSummary getByModelHyperParamSummary(UUID uuid) {
return queryFactory
.select(
Projections.constructor(
HyperSummary.class,
modelHyperParamEntity.uuid,
modelHyperParamEntity.id,
modelHyperParamEntity.hyperVer,
modelHyperParamEntity.backbone,
modelHyperParamEntity.inputSize,
modelHyperParamEntity.cropSize,
modelHyperParamEntity.batchSize))
.from(modelHyperParamEntity)
.where(
modelHyperParamEntity.id.eq(
JPAExpressions.select(modelMasterEntity.hyperParamId)
.from(modelMasterEntity)
.where(modelMasterEntity.uuid.eq(uuid))))
.fetchOne();
}
@Override
public TransferHyperSummary getByModelTransferHyperParamSummary(UUID uuid) {
QModelMasterEntity subMaster = new QModelMasterEntity("subMaster");
QModelHyperParamEntity subHyper = new QModelHyperParamEntity("subHyper");
return queryFactory
.select(
Projections.constructor(
TransferHyperSummary.class,
modelHyperParamEntity.uuid,
modelHyperParamEntity.id,
modelHyperParamEntity.hyperVer,
modelHyperParamEntity.backbone,
modelHyperParamEntity.inputSize,
modelHyperParamEntity.cropSize,
modelHyperParamEntity.batchSize,
subHyper.uuid,
subHyper.id,
subHyper.hyperVer,
subHyper.backbone,
subHyper.inputSize,
subHyper.cropSize,
subHyper.batchSize))
.from(modelMasterEntity)
.innerJoin(modelHyperParamEntity)
.on(modelHyperParamEntity.id.eq(modelMasterEntity.hyperParamId))
.leftJoin(subMaster)
.on(subMaster.id.eq(modelMasterEntity.beforeModelId))
.leftJoin(subHyper)
.on(subHyper.id.eq(subMaster.hyperParamId))
.where(modelMasterEntity.uuid.eq(uuid))
.fetchOne();
}
@Override
public List<MappingDataset> getByModelMappingDataset(UUID uuid) {
return queryFactory
.select(
Projections.constructor(
MappingDataset.class,
modelMasterEntity.id,
datasetEntity.id,
datasetEntity.dataType,
datasetEntity.compareYyyy,
datasetEntity.targetYyyy,
datasetEntity.roundNo,
modelDatasetEntity.buildingCnt,
modelDatasetEntity.containerCnt,
modelDatasetEntity.wasteCnt,
modelDatasetEntity.landCoverCnt))
.from(modelMasterEntity)
.innerJoin(modelDatasetEntity)
.on(modelMasterEntity.id.eq(modelDatasetEntity.model.id))
.innerJoin(modelDatasetMappEntity)
.on(modelMasterEntity.id.eq(modelDatasetMappEntity.modelUid))
.innerJoin(datasetEntity)
.on(modelDatasetMappEntity.datasetUid.eq(datasetEntity.id))
.where(modelMasterEntity.uuid.eq(uuid))
.fetch();
}
@Override
public ModelMasterEntity findByModelByUUID(UUID uuid) {
return queryFactory
.selectFrom(modelMasterEntity)
.where(modelMasterEntity.uuid.eq(uuid))
.fetchOne();
}
@Override
public List<ModelTrainMetrics> getModelTrainMetricResult(UUID uuid) {
ModelMasterEntity modelMasterEntity = findByModelByUUID(uuid);
if (modelMasterEntity == null) {
return List.of();
}
return queryFactory
.select(
Projections.constructor(
ModelTrainMetrics.class,
modelMetricsTrainEntity.epoch,
modelMetricsTrainEntity.iteration,
modelMetricsTrainEntity.loss,
modelMetricsTrainEntity.lr,
modelMetricsTrainEntity.durationTime))
.from(modelMetricsTrainEntity)
.where(modelMetricsTrainEntity.model.id.eq(modelMasterEntity.getId()))
.fetch();
}
@Override
public List<ModelValidationMetrics> getModelValidationMetricResult(UUID uuid) {
ModelMasterEntity modelMasterEntity = findByModelByUUID(uuid);
if (modelMasterEntity == null) {
return List.of();
}
return queryFactory
.select(
Projections.constructor(
ModelValidationMetrics.class,
modelMetricsValidationEntity.epoch,
modelMetricsValidationEntity.aAcc,
modelMetricsValidationEntity.mFscore,
modelMetricsValidationEntity.mPrecision,
modelMetricsValidationEntity.mRecall,
modelMetricsValidationEntity.mIou,
modelMetricsValidationEntity.mAcc,
modelMetricsValidationEntity.changedFscore,
modelMetricsValidationEntity.changedPrecision,
modelMetricsValidationEntity.changedRecall,
modelMetricsValidationEntity.unchangedFscore,
modelMetricsValidationEntity.unchangedPrecision,
modelMetricsValidationEntity.unchangedRecall))
.from(modelMetricsValidationEntity)
.where(modelMetricsValidationEntity.model.id.eq(modelMasterEntity.getId()))
.fetch();
}
@Override
public List<ModelTestMetrics> getModelTestMetricResult(UUID uuid) {
ModelMasterEntity modelMasterEntity = findByModelByUUID(uuid);
if (modelMasterEntity == null) {
return List.of();
}
return queryFactory
.select(
Projections.constructor(
ModelTestMetrics.class,
modelMetricsTestEntity.model1,
modelMetricsTestEntity.tp,
modelMetricsTestEntity.fp,
modelMetricsTestEntity.fn,
modelMetricsTestEntity.precisions,
modelMetricsTestEntity.recall,
modelMetricsTestEntity.f1Score,
modelMetricsTestEntity.accuracy,
modelMetricsTestEntity.iou,
modelMetricsTestEntity.detectionCount,
modelMetricsTestEntity.gtCount))
.from(modelMetricsTestEntity)
.where(modelMetricsTestEntity.model.id.eq(modelMasterEntity.getId()))
.fetch();
}
@Override
public ModelBestEpoch getModelTrainBestEpoch(UUID uuid) {
ModelMasterEntity modelMasterEntity = findByModelByUUID(uuid);
if (modelMasterEntity == null) {
return null;
}
return queryFactory
.select(
Projections.constructor(
ModelBestEpoch.class,
modelMetricsTrainEntity.epoch,
modelMetricsTrainEntity.loss,
modelMetricsValidationEntity.mFscore,
modelMetricsValidationEntity.mPrecision,
modelMetricsValidationEntity.mRecall,
modelMetricsValidationEntity.mIou,
modelMetricsValidationEntity.mAcc))
.from(modelMetricsTrainEntity)
.leftJoin(modelMetricsValidationEntity)
.on(
modelMetricsTrainEntity.model.eq(modelMetricsValidationEntity.model),
modelMetricsTrainEntity.epoch.eq(modelMetricsValidationEntity.epoch))
.where(
modelMetricsTrainEntity.model.id.eq(modelMasterEntity.getId()),
modelMetricsTrainEntity.epoch.eq(modelMasterEntity.getBestEpoch()))
.fetchOne();
}
@Override
public ModelFileInfo getModelTrainFileInfo(UUID uuid) {
return queryFactory
.select(
Projections.constructor(
ModelFileInfo.class,
modelMasterEntity
.packingState
.eq(TrainStatusType.COMPLETED.getId())
.coalesce(false),
modelMasterEntity.modelVer))
.from(modelMasterEntity)
.where(modelMasterEntity.uuid.eq(uuid))
.fetchOne();
}
@Override
public List<ModelProgressStepDto> findModelTrainProgressInfo(UUID uuid) {
ModelMasterEntity entity = findByModelByUUID(uuid);
if (entity == null) {
return List.of();
}
List<ModelProgressStepDto> steps = new ArrayList<>();
// 0단계 : 대기 상태
steps.add(
ModelProgressStepDto.builder()
.step(0)
.status(TrainStatusType.READY.getId())
.startTime(entity.getCreatedDttm())
.endTime(null)
.isError(false)
.build());
// 1단계 : Train/Validation 실행
boolean step1Active =
entity.getStep1StrtDttm() != null
&& !TrainStatusType.READY.getId().equals(entity.getStep1State());
if (step1Active) {
steps.add(
ModelProgressStepDto.builder()
.step(1)
.status(entity.getStep1State())
.startTime(entity.getStep1StrtDttm())
.endTime(entity.getStep1EndDttm())
.isError(TrainStatusType.ERROR.getId().equals(entity.getStep1State()))
.build());
}
// 2단계 : Test 실행
boolean step2Done = entity.getStep2State() != null;
if (step2Done) {
steps.add(
ModelProgressStepDto.builder()
.step(2)
.status(entity.getStep2State())
.startTime(entity.getStep2StrtDttm())
.endTime(entity.getStep2EndDttm())
.isError(TrainStatusType.ERROR.getId().equals(entity.getStep2State()))
.build());
}
// 3단계 : 패키징
boolean step3Done = entity.getPackingState() != null;
if (step3Done) {
steps.add(
ModelProgressStepDto.builder()
.step(3)
.status(entity.getPackingState())
.startTime(entity.getPackingStrtDttm())
.endTime(entity.getPackingEndDttm())
.isError(TrainStatusType.ERROR.getId().equals(entity.getPackingState()))
.build());
}
return steps;
}
@Override
public ModelMasterEntity findByModelBeforeId(Long beforeModelId) {
return queryFactory
.selectFrom(modelMasterEntity)
.where(modelMasterEntity.id.eq(beforeModelId))
.fetchOne();
}
}

Some files were not shown because too many files have changed in this diff Show More