352 Commits

Author SHA1 Message Date
3d2a4049d3 시스템 사용율 모니터링 기능 로그 수정 2026-03-19 17:38:30 +09:00
0cbaf53e86 시스템 사용율 모니터링 기능 추가 2026-03-19 17:08:37 +09:00
80fd2bda3e 시스템 사용율 모니터링 기능 테스트 2026-03-19 16:52:26 +09:00
fb647e5991 시스템 사용율 모니터링 기능 테스트 2026-03-19 16:36:39 +09:00
87575a62f7 시스템 사용율 모니터링 기능 테스트 2026-03-19 16:27:05 +09:00
246c11f8b0 시스템 사용율 모니터링 기능 테스트 2026-03-19 16:12:27 +09:00
2c1f9bdf5c 시스템 사용율 모니터링 기능 테스트 2026-03-19 15:46:23 +09:00
5799f7dfb2 시스템 사용율 모니터링 기능 테스트 2026-03-19 15:41:18 +09:00
9f428e9572 시스템 사용율 모니터링 기능 테스트 2026-03-19 15:37:26 +09:00
904968a1be 시스템 사용율 모니터링 기능 테스트 2026-03-19 15:25:20 +09:00
4b44be6a29 시스템 사용율 모니터링 기능 테스트 2026-03-19 15:25:11 +09:00
f9f0662f8e 시스템 사용율 모니터링 기능 테스트 2026-03-19 15:24:03 +09:00
7416327cc3 ai 학습실행 run command 수정 2026-03-11 10:18:33 +09:00
da31bd9d99 ai 학습실행 run command 수정 2026-03-10 23:09:39 +09:00
f3e5347335 ai 학습실행 run command 수정 2026-03-10 22:48:10 +09:00
7d5581f60c 데이터셋 삭제 플래그 추가 2026-03-10 19:09:00 +09:00
b4428217ea hello 2026-03-10 18:01:03 +09:00
8a63fdacdd confict 2026-03-10 17:27:14 +09:00
cb2e42143a confict 2026-03-10 17:23:27 +09:00
997e85c0cc 운영환경처리 2026-03-10 17:22:27 +09:00
731ca59475 심볼릭 링크로 수정 2026-03-10 17:16:54 +09:00
fe6d37456d 하드링크 실패 시 심볼릭 링크로 만들어보기 2026-03-10 16:56:35 +09:00
6c98a48a5d 에러 수정 2026-03-10 16:34:02 +09:00
81b69caa99 운영환경처리 2026-03-10 16:00:23 +09:00
7fce070686 spotless 2026-03-10 15:25:32 +09:00
8d83505ee7 운영환경처리 2026-03-10 15:00:31 +09:00
0ff38b24d4 Merge branch 'feat/training_260202' into develop
# Conflicts:
#	src/main/java/com/kamco/cd/training/train/service/JobRecoveryOnStartupService.java
2026-03-10 14:22:14 +09:00
265813e6f7 Merge branch 'feat/training_260202' of https://kamco.git.gs.dabeeo.com/MVPTeam/kamco-train-api into feat/training_260202 2026-03-10 14:19:40 +09:00
8190a6e9c8 unzip 2026-03-10 14:19:22 +09:00
5c082f7c9d 운영환경처리 2026-03-10 08:39:42 +09:00
43d0e55cb7 Merge branch 'develop' of https://kamco.git.gs.dabeeo.com/MVPTeam/kamco-train-api into develop 2026-03-04 06:12:27 +09:00
df3bedfbda prod 2026-03-04 05:35:21 +09:00
c26a48d07d prod 2026-03-04 05:25:04 +09:00
1c7c213977 Merge pull request '학습 실패 처리 수정' (#155) from feat/training_260303 into develop
Reviewed-on: #155
2026-03-04 01:43:56 +09:00
6583a45abd 학습 실패 처리 수정 2026-03-04 01:43:34 +09:00
b15f77d894 Merge pull request '학습 실패 처리 수정' (#154) from feat/training_260303 into develop
Reviewed-on: #154
2026-03-04 01:34:40 +09:00
3bcd99f0db 학습 실패 처리 수정 2026-03-04 01:34:16 +09:00
5513cd60a0 Merge pull request '리커버리 수정' (#153) from feat/training_260303 into develop
Reviewed-on: #153
2026-03-04 01:18:19 +09:00
7b35d26a13 리커버리 수정 2026-03-04 01:18:04 +09:00
d92ff88ef7 Merge pull request '리커버리 수정' (#152) from feat/training_260303 into develop
Reviewed-on: #152
2026-03-04 01:15:04 +09:00
bfcddd0327 리커버리 수정 2026-03-04 01:14:35 +09:00
e193330f99 Merge pull request '리커버리 수정' (#151) from feat/training_260303 into develop
Reviewed-on: #151
2026-03-04 01:00:45 +09:00
6c0184597d 리커버리 수정 2026-03-04 01:00:26 +09:00
1e8a8d8dad Merge pull request '리커버리 수정 테스트 로그 추가,' (#150) from feat/training_260303 into develop
Reviewed-on: #150
2026-03-04 00:44:03 +09:00
eb7680b952 리커버리 수정 테스트 로그 추가, 2026-03-04 00:43:20 +09:00
2c357ebf27 Merge pull request '리커버리 수정 테스트,,' (#149) from feat/training_260303 into develop
Reviewed-on: #149
2026-03-04 00:30:36 +09:00
0daaa1c8cb 리커버리 수정 테스트,, 2026-03-04 00:30:17 +09:00
96383595df Merge pull request '리커버리 수정' (#148) from feat/training_260303 into develop
Reviewed-on: #148
2026-03-04 00:22:09 +09:00
c5e03f7ca8 리커버리 수정 2026-03-04 00:21:51 +09:00
df2acc4dfb Merge pull request '리커버리 수정' (#147) from feat/training_260303 into develop
Reviewed-on: #147
2026-03-04 00:06:21 +09:00
693de354d2 리커버리 수정 2026-03-04 00:06:04 +09:00
62cdc5015e Merge pull request '학습 리커버리 테스트' (#146) from feat/training_260303 into develop
Reviewed-on: #146
2026-03-03 23:45:46 +09:00
5c11263d55 랃그 2026-03-03 23:45:10 +09:00
dd5031ae3a Merge pull request 'log.info 추가' (#145) from feat/training_260303 into develop
Reviewed-on: #145
2026-03-03 23:40:44 +09:00
c369e01ada log.info 추가 2026-03-03 23:40:18 +09:00
54991622f1 log.info 추가 2026-03-03 23:31:25 +09:00
e9f8bb37fa spotless 적용 2026-03-03 23:06:45 +09:00
17d69486ec spotless 적용 2026-03-03 23:04:55 +09:00
f3c822587f spotless 적용 2026-03-03 23:02:53 +09:00
948f1061da Merge pull request 'spotless 적용' (#144) from feat/training_260202 into develop
Reviewed-on: #144
2026-03-03 23:01:39 +09:00
335f0dbb9b spotless 적용 2026-03-03 23:01:22 +09:00
42438b3cd5 Merge pull request '하드링크 수정' (#143) from feat/training_260202 into develop
Reviewed-on: #143
2026-03-03 22:51:50 +09:00
69eaba1a83 하드링크 수정 2026-03-03 22:51:10 +09:00
524ae200b0 prod 2026-03-03 08:44:31 +09:00
4f763d3c2e prod 2026-03-03 08:44:01 +09:00
9ebf525387 prod 2026-03-03 08:42:16 +09:00
4ab672a96e 중복 경고 2026-02-28 01:56:48 +09:00
7d2a367e3f Merge pull request '리커버리 삭제' (#142) from feat/training_260202 into develop
Reviewed-on: #142
2026-02-28 01:24:49 +09:00
365ad81cad 리커버리 삭제 2026-02-28 01:24:34 +09:00
67a67749c3 Merge pull request '리커버리 추가' (#141) from feat/training_260202 into develop
Reviewed-on: #141
2026-02-28 01:02:10 +09:00
9dfa54fbf9 리커버리 추가 2026-02-28 01:01:38 +09:00
251307b5c9 Merge pull request '하드링크 수정' (#140) from feat/training_260202 into develop
Reviewed-on: #140
2026-02-27 23:31:24 +09:00
12f6bb7154 하드링크 수정 2026-02-27 23:31:04 +09:00
8423a03d31 Merge pull request '하드링크 로그 추가' (#139) from feat/training_260202 into develop
Reviewed-on: #139
2026-02-27 23:12:17 +09:00
aa3af4e9d0 하드링크 로그 추가 2026-02-27 23:12:00 +09:00
d6cdf6b690 Merge pull request '하드링크 로그 추가' (#138) from feat/training_260202 into develop
Reviewed-on: #138
2026-02-27 22:51:51 +09:00
7ca37bf1e4 하드링크 로그 추가 2026-02-27 22:51:27 +09:00
9cfa299e58 Merge pull request 'feat/training_260202' (#137) from feat/training_260202 into develop
Reviewed-on: #137
2026-02-24 16:55:35 +09:00
901dde066d 하이파라미터 상세조회 수정 2026-02-24 16:54:58 +09:00
cb0a38274a val_interval 기본값 1로 수정 2026-02-24 16:24:16 +09:00
b8194df9ae 학습 실패여부 확인 기능 추가 2026-02-24 15:43:35 +09:00
a137e71420 Merge pull request 'feat/training_260202' (#136) from feat/training_260202 into develop
Reviewed-on: #136
2026-02-24 15:11:12 +09:00
7c5f07683e 학습 실패여부 확인 기능 추가 2026-02-24 15:10:48 +09:00
159fb281d4 최근 사용일시 업데이트 2026-02-23 15:40:24 +09:00
97192ff811 최근 사용일시 업데이트 2026-02-23 15:39:25 +09:00
4f3fb675be 하이퍼 파라미터 사용회수 카운트 기능 추가 및 조회 수정 2026-02-23 15:37:28 +09:00
e6caea05b3 하이퍼 파라미터 사용회수 카운트 기능 추가 및 조회 수정 2026-02-23 15:19:40 +09:00
f08f80622f Merge pull request 'feat/training_260202' (#135) from feat/training_260202 into develop
Reviewed-on: #135
2026-02-23 14:31:11 +09:00
fd63824edc 하이퍼 파라미터 미사용 컬럼제거, 사용횟수 컬럼 추가 2026-02-23 14:30:29 +09:00
8a44df26b8 하이퍼 파라미터 상세조회 삭제여부 조건 제거 2026-02-23 14:17:34 +09:00
cb97c5e59e 하이퍼 파라미터 dto 주석 수정 2026-02-23 14:13:25 +09:00
8f75b16dc6 학습실행 주석 추가 2026-02-23 12:30:54 +09:00
e565fd7a34 Merge pull request '전이학습 상세 수정' (#134) from feat/training_260202 into develop
Reviewed-on: #134
2026-02-20 18:34:58 +09:00
c2978e41c2 전이학습 상세 수정 2026-02-20 18:34:32 +09:00
8c45b39dcc Merge pull request 'feat/training_260202' (#133) from feat/training_260202 into develop
Reviewed-on: #133
2026-02-20 18:22:45 +09:00
07429dbe8e 전이학습 상세 수정 2026-02-20 18:22:19 +09:00
83859bb9fe 전이학습 상세 - before dataset 추가 2026-02-20 16:05:29 +09:00
b119f333ac Merge pull request 'best epoch 파일 선택 수정' (#132) from feat/training_260202 into develop
Reviewed-on: #132
2026-02-20 15:41:58 +09:00
564a99448c best epoch 파일 선택 수정 2026-02-20 15:41:34 +09:00
bbe04ee458 Merge pull request 'best epoch 파일 선택 수정' (#131) from feat/training_260202 into develop
Reviewed-on: #131
2026-02-20 15:31:50 +09:00
38ae6e5575 best epoch 파일 선택 수정 2026-02-20 15:31:33 +09:00
fab3c83a69 Merge pull request 'best epoch 파일 선택 수정' (#130) from feat/training_260202 into develop
Reviewed-on: #130
2026-02-20 15:15:39 +09:00
40fe98ae0c best epoch 파일 선택 수정 2026-02-20 15:15:12 +09:00
fb87a0f32f Merge pull request '중복 수정 제거' (#129) from feat/training_260202 into develop
Reviewed-on: #129
2026-02-20 14:31:15 +09:00
255ff10a56 Merge remote-tracking branch 'origin/feat/training_260202' into feat/training_260202 2026-02-20 14:30:42 +09:00
f674f73330 중복 수정 제거 2026-02-20 14:30:34 +09:00
63794ec4ec Merge pull request 'feat/training_260202' (#128) from feat/training_260202 into develop
Reviewed-on: #128
2026-02-20 14:25:13 +09:00
db2bc32e7d test metrics insert 로직 수정 2026-02-20 14:24:23 +09:00
37786a1e44 선택한 테스트 에폭 로그 추가 2026-02-20 14:13:49 +09:00
901ea83fb7 test 선택한 에폭 log 확인 추가 2026-02-20 14:13:22 +09:00
bf6dc9740f Merge pull request 'tmp 하드링크 수정' (#127) from feat/training_260202 into develop
Reviewed-on: #127
2026-02-20 13:37:04 +09:00
832e1b5681 tmp 하드링크 수정 2026-02-20 13:36:48 +09:00
a23bc8dd67 Merge pull request 'tmp 하드링크 수정' (#126) from feat/training_260202 into develop
Reviewed-on: #126
2026-02-20 12:30:23 +09:00
4f16355cda tmp 하드링크 수정 2026-02-20 12:29:57 +09:00
13023a06cc Merge pull request 'feat/training_260202' (#125) from feat/training_260202 into develop
Reviewed-on: #125
2026-02-20 12:23:37 +09:00
df46a8f79f Merge remote-tracking branch 'origin/feat/training_260202' into feat/training_260202 2026-02-20 12:21:50 +09:00
fcd48831c5 tmp 하드링크 수정 2026-02-20 12:21:41 +09:00
28b50bd949 Merge pull request 'test json 수정' (#124) from feat/training_260202 into develop
Reviewed-on: #124
2026-02-20 12:20:36 +09:00
62c9d73b94 test json 수정 2026-02-20 12:20:15 +09:00
78ab928459 Merge pull request 'ing-cnt 로직에 step2도 추가, transactional' (#123) from feat/training_260202 into develop
Reviewed-on: #123
2026-02-20 12:05:40 +09:00
68c0e634c5 ing-cnt 로직에 step2도 추가, transactional 2026-02-20 12:05:20 +09:00
ae3601cff5 Merge pull request '비밀번호 변경 security 로직 수정' (#122) from feat/training_260202 into develop
Reviewed-on: #122
2026-02-20 11:37:08 +09:00
ad421e3c74 비밀번호 변경 security 로직 수정 2026-02-20 11:36:21 +09:00
5f62f4a209 Merge pull request 'test 실행 시 회차별 데이터 적재하기' (#121) from feat/training_260202 into develop
Reviewed-on: #121
2026-02-19 18:19:40 +09:00
46db1512a6 test 실행 시 회차별 데이터 적재하기 2026-02-19 18:18:12 +09:00
29bf155b4f Merge pull request 'LogErrorLevel -> CodeExpose 추가' (#120) from feat/training_260202 into develop
Reviewed-on: #120
2026-02-19 17:35:45 +09:00
2034a8fcb2 LogErrorLevel -> CodeExpose 추가 2026-02-19 17:35:15 +09:00
da03f8b749 Merge pull request '모델학습관리 > 모델별 진행 상황 API 추가' (#119) from feat/training_260202 into develop
Reviewed-on: #119
2026-02-19 17:17:46 +09:00
bf212842d8 모델학습관리 > 모델별 진행 상황 API 추가 2026-02-19 17:17:11 +09:00
6a2deff93b Merge pull request '모델학습관리 > 목록 API 메모,작성자 추가로 인한 수정' (#118) from feat/training_260202 into develop
Reviewed-on: #118
2026-02-19 15:35:03 +09:00
d2ca94ea55 모델학습관리 > 목록 API 메모,작성자 추가로 인한 수정 2026-02-19 15:34:18 +09:00
b0a99afcd3 Merge pull request '모델학습 2단계 패키징 시작,종료일시,상태 로직 추가' (#117) from feat/training_260202 into develop
Reviewed-on: #117
2026-02-19 14:44:12 +09:00
5ddf6dfeeb 모델학습 2단계 패키징 시작,종료일시,상태 로직 추가 2026-02-19 14:43:14 +09:00
eedf72d7aa Merge pull request '공통코드 common-code 로 prefix 변경' (#116) from feat/training_260202 into develop
Reviewed-on: #116
2026-02-19 11:39:23 +09:00
5e13c0b396 공통코드 common-code 로 prefix 변경 2026-02-19 11:38:56 +09:00
25e9941464 Merge pull request '로그관리 로직 커밋' (#115) from feat/training_260202 into develop
Reviewed-on: #115
2026-02-19 11:16:40 +09:00
435f60dcac 로그관리 로직 커밋 2026-02-19 11:13:40 +09:00
a0da0392cf Merge pull request '압축해제 시, 동일 폴더가 있으면 삭제 후 재업로드' (#114) from feat/training_260202 into develop
Reviewed-on: #114
2026-02-18 16:37:13 +09:00
5f5eabca19 압축해제 시, 동일 폴더가 있으면 삭제 후 재업로드 2026-02-18 16:36:46 +09:00
a3ebee12b5 Merge pull request 'feat/training_260202' (#113) from feat/training_260202 into develop
Reviewed-on: #113
2026-02-18 16:29:55 +09:00
413631840f 학습데이터 다운로드 security 제외하기 2026-02-18 16:29:28 +09:00
c7f63d1ad1 압축 해제한 폴더의 갯수 맞는지 log 찍기 + 갯수 맞지 않으면 exception 리턴 2026-02-18 16:22:32 +09:00
c5b14ca09d Merge pull request '업로드 시 uid로 중복체크 -> 삭제인 row는 제외하기' (#112) from feat/training_260202 into develop
Reviewed-on: #112
2026-02-18 15:40:38 +09:00
7529d23488 업로드 시 uid로 중복체크 -> 삭제인 row는 제외하기 2026-02-18 15:40:14 +09:00
44b3b857b1 Merge pull request 'feat/training_260202' (#111) from feat/training_260202 into develop
Reviewed-on: #111
2026-02-18 15:29:25 +09:00
cb3e51d712 업로드 시 exception 메세지 처리, 에폭 10 이상으로 실행되게 수정 2026-02-18 15:28:29 +09:00
99a4597b5f train 결과 +1 했던 거 제거하기 2026-02-18 14:50:53 +09:00
63124455fd Merge pull request '1단계 실행 시, 시작시간 update 추가' (#110) from feat/training_260202 into develop
Reviewed-on: #110
2026-02-18 13:06:33 +09:00
d9da0d4610 1단계 실행 시, 시작시간 update 추가 2026-02-18 13:05:59 +09:00
e75ea8d8a5 Merge pull request '하이퍼 파라미터 수정' (#109) from feat/training_260202 into develop
Reviewed-on: #109
2026-02-13 15:00:50 +09:00
22c481556c 하이퍼 파라미터 수정 2026-02-13 15:00:30 +09:00
31ac4209c3 Merge pull request '하이퍼 파라미터 수정' (#108) from feat/training_260202 into develop
Reviewed-on: #108
2026-02-13 14:47:19 +09:00
0798b352c7 하이퍼 파라미터 수정 2026-02-13 14:46:59 +09:00
df09935789 Merge pull request '하이퍼 파라미터 수정' (#107) from feat/training_260202 into develop
Reviewed-on: #107
2026-02-13 14:42:54 +09:00
5b074bdb81 하이퍼 파라미터 수정 2026-02-13 14:42:39 +09:00
bb15b1b0f2 Merge pull request '하이퍼 파라미터 수정' (#106) from feat/training_260202 into develop
Reviewed-on: #106
2026-02-13 14:38:01 +09:00
28919345c2 하이퍼 파라미터 수정 2026-02-13 14:37:44 +09:00
f4d491ed94 Merge pull request '하이퍼 파라미터 수정' (#105) from feat/training_260202 into develop
Reviewed-on: #105
2026-02-13 14:31:13 +09:00
aa0552aaa7 하이퍼 파라미터 수정 2026-02-13 14:30:45 +09:00
96cb7d2f23 Merge pull request '사용가능 용량 API 수정' (#104) from feat/training_260202 into develop
Reviewed-on: #104
2026-02-13 14:19:26 +09:00
5d0aca14a6 사용가능 용량 API 수정 2026-02-13 14:19:09 +09:00
cc6305b0df Merge pull request '이어하기 수정' (#103) from feat/training_260202 into develop
Reviewed-on: #103
2026-02-13 14:08:51 +09:00
af8d59ddfa 이어하기 수정 2026-02-13 14:08:35 +09:00
3916b13876 Merge pull request '이어하기 수정' (#102) from feat/training_260202 into develop
Reviewed-on: #102
2026-02-13 14:04:52 +09:00
4f24e09c57 이어하기 수정 2026-02-13 14:04:33 +09:00
ee4a06df30 Merge pull request '이어하기 수정' (#101) from feat/training_260202 into develop
Reviewed-on: #101
2026-02-13 13:57:48 +09:00
4da477706f 이어하기 수정 2026-02-13 13:57:01 +09:00
bb5ff7c3cd Merge pull request '이어하기 로그 수정' (#100) from feat/training_260202 into develop
Reviewed-on: #100
2026-02-13 13:27:08 +09:00
a070566048 이어하기 로그 수정 2026-02-13 13:26:54 +09:00
312a96dda1 Merge pull request '트랜젝션처리 임시폴더 uid업데이트' (#99) from feat/training_260202 into develop
Reviewed-on: #99
2026-02-13 13:24:10 +09:00
a5b3ae613f 트랜젝션처리 임시폴더 uid업데이트 2026-02-13 13:23:53 +09:00
e38231e06d Merge pull request '트랜젝션처리 임시폴더 uid업데이트' (#98) from feat/training_260202 into develop
Reviewed-on: #98
2026-02-13 13:17:41 +09:00
979af088be 트랜젝션처리 임시폴더 uid업데이트 2026-02-13 13:17:27 +09:00
bf6e45d706 Merge pull request 'feat/training_260202' (#97) from feat/training_260202 into develop
Reviewed-on: #97
2026-02-13 12:53:32 +09:00
e5a1cab36b Merge branch 'feat/training_260202' of https://kamco.git.gs.dabeeo.com/MVPTeam/kamco-train-api into feat/training_260202 2026-02-13 12:53:04 +09:00
bb67996742 text 2026-02-13 12:53:00 +09:00
1981d6d1ce Merge remote-tracking branch 'origin/feat/training_260202' into feat/training_260202 2026-02-13 12:53:00 +09:00
47f4ffd4db 트랜젝션처리 임시폴더 uid업데이트 2026-02-13 12:52:11 +09:00
7fa8921a25 Merge pull request 'flush 추가해보기' (#96) from feat/training_260202 into develop
Reviewed-on: #96
2026-02-13 12:47:11 +09:00
195856b846 flush 추가해보기 2026-02-13 12:46:54 +09:00
7c940351d9 Merge pull request '트랜젝션처리 임시폴더 uid업데이트' (#95) from feat/training_260202 into develop
Reviewed-on: #95
2026-02-13 12:39:21 +09:00
124da48e51 트랜젝션처리 임시폴더 uid업데이트 2026-02-13 12:38:55 +09:00
6b834da912 Merge pull request 'feat/training_260202' (#94) from feat/training_260202 into develop
Reviewed-on: #94
2026-02-13 12:25:26 +09:00
02724e9508 주석한거 원복 2026-02-13 12:24:50 +09:00
7ed91ccab9 Merge branch 'feat/training_260202' of https://kamco.git.gs.dabeeo.com/MVPTeam/kamco-train-api into feat/training_260202 2026-02-13 12:23:12 +09:00
a7c13b985d responsePath 셋팅 삭제 2026-02-13 12:23:01 +09:00
25aaa97d65 Merge pull request '트랜젝션처리 임시폴더 uid업데이트' (#93) from feat/training_260202 into develop
Reviewed-on: #93
2026-02-13 12:21:11 +09:00
352a28b87f 트랜젝션처리 임시폴더 uid업데이트 2026-02-13 12:20:48 +09:00
da9d47ae4a Merge pull request '주석 처리' (#92) from feat/training_260202 into develop
Reviewed-on: #92
2026-02-13 12:10:46 +09:00
bf8515163c 주석 처리 2026-02-13 12:10:08 +09:00
7d6a77bf2a Merge pull request '트랜젝션처리 임시폴더 uid업데이트' (#91) from feat/training_260202 into develop
Reviewed-on: #91
2026-02-13 12:01:01 +09:00
26828d0968 add log 2026-02-13 12:00:54 +09:00
2691f6ce16 트랜젝션처리 임시폴더 uid업데이트 2026-02-13 12:00:42 +09:00
e2dbae15c0 Merge pull request '트랜젝션처리 임시폴더 uid업데이트' (#90) from feat/training_260202 into develop
Reviewed-on: #90
2026-02-13 11:58:55 +09:00
b246034632 Merge pull request '트랜젝션처리 임시폴더 uid업데이트' (#89) from feat/training_260202 into develop
Reviewed-on: #89
2026-02-13 11:58:25 +09:00
7e5aa5e713 트랜젝션처리 임시폴더 uid업데이트 2026-02-13 11:57:48 +09:00
060a815e1c 트랜젝션처리 임시폴더 uid업데이트 2026-02-13 11:55:35 +09:00
687ea82d78 Merge pull request 'feat/training_260202' (#88) from feat/training_260202 into develop
Reviewed-on: #88
2026-02-13 10:51:07 +09:00
1eb4d04779 Merge branch 'feat/training_260202' of https://kamco.git.gs.dabeeo.com/MVPTeam/kamco-train-api into feat/training_260202 2026-02-13 10:50:35 +09:00
f30c0c6d45 다운로드 시 Access-Control-Expose-Headers 추가 2026-02-13 10:50:28 +09:00
12994aab60 파일 count 기능 추가 2026-02-13 10:44:49 +09:00
4ac0f19908 Merge pull request '파일 count 기능 추가' (#87) from feat/training_260202 into develop
Reviewed-on: #87
2026-02-13 10:38:48 +09:00
11d3afe295 파일 count 기능 추가 2026-02-13 10:38:24 +09:00
9e5e7595eb Merge pull request '학습실행 step1 할 때 best epoch 업데이트' (#86) from feat/training_260202 into develop
Reviewed-on: #86
2026-02-13 10:18:26 +09:00
1e62a8b097 학습실행 step1 할 때 best epoch 업데이트 2026-02-13 10:15:04 +09:00
9cd9274e99 Merge pull request '학습데이터 목록 파일 단위 MB 나오게 하기' (#85) from feat/training_260202 into develop
Reviewed-on: #85
2026-02-13 09:48:08 +09:00
26a4623aa8 학습데이터 목록 파일 단위 MB 나오게 하기 2026-02-13 09:42:36 +09:00
5d82f3ecfe Merge pull request 'tmp 파일 링크 수정' (#84) from feat/training_260202 into develop
Reviewed-on: #84
2026-02-13 09:11:05 +09:00
ce6e4f5aea tmp 파일 링크 수정 2026-02-13 09:10:45 +09:00
2ce249ab33 Merge pull request 'tmp 파일 링크 수정' (#83) from feat/training_260202 into develop
Reviewed-on: #83
2026-02-13 08:44:37 +09:00
c2215836c0 tmp 파일 링크 수정 2026-02-13 08:44:17 +09:00
e34bf68de0 Merge pull request 'tmp 파일 링크 수정' (#82) from feat/training_260202 into develop
Reviewed-on: #82
2026-02-13 08:33:58 +09:00
8c19c996f7 tmp 파일 링크 수정 2026-02-13 08:33:36 +09:00
862bda0cb9 Merge pull request '이어하기 수정' (#81) from feat/training_260202 into develop
Reviewed-on: #81
2026-02-12 23:02:12 +09:00
b5ce3ab1fb 이어하기 수정 2026-02-12 23:01:56 +09:00
90f7b17d07 Merge pull request '학습데이터 다운로드 파일 정보 API 추가' (#80) from feat/training_260202 into develop
Reviewed-on: #80
2026-02-12 22:58:47 +09:00
e1ceb769dd 학습데이터 다운로드 파일 정보 API 추가 2026-02-12 22:47:13 +09:00
2128baa46a Merge pull request 'feat/training_260202' (#79) from feat/training_260202 into develop
Reviewed-on: #79
2026-02-12 22:26:26 +09:00
4219b88fb3 학습데이터 다운로드 API 추가 2026-02-12 22:25:55 +09:00
4f94c99b64 이어하기 수정 2026-02-12 22:09:54 +09:00
875c30f467 Merge pull request 'feat/training_260202' (#78) from feat/training_260202 into develop
Reviewed-on: #78
2026-02-12 21:52:16 +09:00
d42e1afbd4 스케줄러 api 수동 호출 2026-02-12 21:51:53 +09:00
b3b8016673 csv 결과 받아오는 것 변경 2026-02-12 21:45:38 +09:00
2b29cd1ac6 Merge pull request '파라미터 변경' (#77) from feat/training_260202 into develop
Reviewed-on: #77
2026-02-12 21:30:18 +09:00
79e8259f28 파라미터 변경 2026-02-12 21:30:03 +09:00
9206fff5d0 Merge pull request 'feat/training_260202' (#76) from feat/training_260202 into develop
Reviewed-on: #76
2026-02-12 21:17:33 +09:00
032c82c2f0 file 경로 넣기 2026-02-12 21:17:10 +09:00
6204a6e5fa Merge remote-tracking branch 'origin/develop' into feat/training_260202
# Conflicts:
#	src/main/resources/application-prod.yml
2026-02-12 21:15:21 +09:00
4d9c9a86b4 패키징 zip파일 만들기 커밋 2026-02-12 21:09:40 +09:00
83204abfe9 Merge pull request '성공시 csv 파일 테이블에 저장 연결' (#74) from feat/training_260202 into develop
Reviewed-on: #74
2026-02-12 21:01:48 +09:00
5b682c1386 성공시 csv 파일 테이블에 저장 연결 2026-02-12 21:01:24 +09:00
452494d44d Merge pull request '테스트 실행 경로 수정' (#73) from feat/training_260202 into develop
Reviewed-on: #73
2026-02-12 20:49:30 +09:00
8ada26448b 테스트 실행 경로 수정 2026-02-12 20:49:14 +09:00
e442f105bc Merge pull request '도커명 변경' (#72) from feat/training_260202 into develop
Reviewed-on: #72
2026-02-12 20:37:00 +09:00
5e0a771848 도커명 변경 2026-02-12 20:36:38 +09:00
b4c2685059 Merge pull request '도커 설정 추가' (#71) from feat/training_260202 into develop
Reviewed-on: #71
2026-02-12 20:25:16 +09:00
e238f3ca88 도커 설정 추가 2026-02-12 20:24:57 +09:00
97b06eb3b3 Merge pull request '임시파일생성 경로 수정' (#70) from feat/training_260202 into develop
Reviewed-on: #70
2026-02-12 20:03:32 +09:00
ad32ca18ca 임시파일생성 경로 수정 2026-02-12 20:03:03 +09:00
98a1283ebe Merge pull request '임시파일생성 경로 수정' (#69) from feat/training_260202 into develop
Reviewed-on: #69
2026-02-12 19:36:06 +09:00
a10fccaae3 임시파일생성 경로 수정 2026-02-12 19:35:47 +09:00
c3c9191d9d Merge pull request 'hyperparam_with_modeltype' (#68) from feat/dean/hyperparam_with_modelType-bug into develop
Reviewed-on: #68
2026-02-12 19:30:29 +09:00
9fd5a15a72 hyperparam_with_modeltype 2026-02-12 19:30:08 +09:00
12f9de7367 hyperparam_with_modeltype 2026-02-12 19:16:24 +09:00
5455da1e96 hyperparam_with_modeltype 2026-02-12 19:16:13 +09:00
9e803661cd Merge pull request 'feat/training_260202' (#67) from feat/training_260202 into develop
Reviewed-on: #67
2026-02-12 19:14:39 +09:00
b0cf9e77ec Merge branch 'develop' of https://kamco.git.gs.dabeeo.com/MVPTeam/kamco-train-api into develop 2026-02-12 19:14:10 +09:00
c92426aefc Merge branch 'feat/training_260202' of https://kamco.git.gs.dabeeo.com/MVPTeam/kamco-train-api into feat/training_260202 2026-02-12 19:14:09 +09:00
d5b2b8ecec hyperparam_with_modeltype 2026-02-12 19:14:01 +09:00
6185a18a7c 모델목록 검색 조건 상태값 변경 2026-02-12 19:13:56 +09:00
49d3e37458 Merge pull request '임시파일생성 경로 수정' (#66) from feat/training_260202 into develop
Reviewed-on: #66
2026-02-12 19:12:37 +09:00
1fb10830b9 임시파일생성 경로 수정 2026-02-12 19:11:51 +09:00
d7766edd24 Merge pull request 'return 형식 수정' (#65) from feat/training_260202 into develop
Reviewed-on: #65
2026-02-12 18:59:37 +09:00
0bc4453c9c hyperparam_with_modeltype 2026-02-12 18:56:32 +09:00
ae0d30e5da return 형식 수정 2026-02-12 18:55:42 +09:00
37d776dd2c Merge pull request 'hyperparam_with_modeltype' (#64) from feat/dean/hyperparam_with_modelType into develop
Reviewed-on: #64
2026-02-12 18:50:32 +09:00
0c34ea7dcb hyperparam_with_modeltype 2026-02-12 18:48:14 +09:00
3106d36431 Merge pull request '업로드 시 같은 uid로 업로드하지 못하게 조건 추가' (#63) from feat/training_260202 into develop
Reviewed-on: #63
2026-02-12 18:44:49 +09:00
ed48f697a4 업로드 시 같은 uid로 업로드하지 못하게 조건 추가 2026-02-12 18:44:04 +09:00
da92b28d97 Merge pull request '임시파일생성 소프트링크에서 하드링크로 변경' (#62) from feat/training_260202 into develop
Reviewed-on: #62
2026-02-12 18:20:30 +09:00
6c865d26fd 임시파일생성 소프트링크에서 하드링크로 변경 2026-02-12 18:18:44 +09:00
e3f00876f1 Merge pull request '문제되는 하이퍼파라미터 주석처리' (#61) from feat/training_260202 into develop
Reviewed-on: #61
2026-02-12 17:53:11 +09:00
16e156b5b4 문제되는 하이퍼파라미터 주석처리 2026-02-12 17:52:42 +09:00
60962bbc75 Merge pull request '학습실행 mount 경로 수정' (#60) from feat/training_260202 into develop
Reviewed-on: #60
2026-02-12 17:44:15 +09:00
6a939118ff 임시폴더생성 api 추가 2026-02-12 17:43:41 +09:00
64d37dcc08 Merge pull request '임시폴더생성 api 추가' (#59) from feat/training_260202 into develop
Reviewed-on: #59
2026-02-12 17:23:53 +09:00
0c0ae16c2b 임시폴더생성 api 추가 2026-02-12 17:23:34 +09:00
a2490f30e6 Merge pull request '임시폴더생성 api 수정' (#58) from feat/training_260202 into develop
Reviewed-on: #58
2026-02-12 17:14:52 +09:00
953f95aed6 임시폴더생성 api 추가 2026-02-12 17:14:26 +09:00
bd04e1f4e8 Merge pull request '임시폴더생성 api 추가' (#57) from feat/training_260202 into develop
Reviewed-on: #57
2026-02-12 17:03:39 +09:00
85633c8bab 임시폴더생성 api 추가 2026-02-12 17:03:21 +09:00
5fc15937c0 Merge pull request 'feat/training_260202' (#56) from feat/training_260202 into develop
Reviewed-on: #56
2026-02-12 17:00:08 +09:00
8b3940b446 Merge remote-tracking branch 'origin/feat/training_260202' into feat/training_260202 2026-02-12 16:59:44 +09:00
201cfefb6b 임시폴더생성 api 추가 2026-02-12 16:59:39 +09:00
9958b0999a csv 읽는 경로 수정하기, 변수명 수정 2026-02-12 16:58:28 +09:00
3547c28361 Merge pull request 'feat/training_260202' (#55) from feat/training_260202 into develop
Reviewed-on: #55
2026-02-12 16:56:23 +09:00
6c70bfed18 Merge remote-tracking branch 'origin/feat/training_260202' into feat/training_260202
# Conflicts:
#	src/main/java/com/kamco/cd/training/postgres/core/ModelTrainMngCoreService.java
2026-02-12 16:55:52 +09:00
95a75e63f4 임시폴더생성 api 추가 2026-02-12 16:55:10 +09:00
2a1dbee290 Merge pull request '모델학습 1단계 실행중인 것이 있는지 count API' (#54) from feat/training_260202 into develop
Reviewed-on: #54
2026-02-12 16:51:09 +09:00
384a321bf3 모델학습 1단계 실행중인 것이 있는지 count API 2026-02-12 16:50:40 +09:00
f4e97d389b Merge pull request 'file 확인 API 수정' (#53) from feat/training_260202 into develop
Reviewed-on: #53
2026-02-12 16:42:20 +09:00
590810ff0a file 확인 API 수정 2026-02-12 16:41:40 +09:00
a01c872982 Merge pull request 'feat/training_260202' (#52) from feat/training_260202 into develop
Reviewed-on: #52
2026-02-12 16:15:11 +09:00
905a245070 Merge branch 'feat/training_260202' of https://kamco.git.gs.dabeeo.com/MVPTeam/kamco-train-api into feat/training_260202 2026-02-12 16:14:45 +09:00
860ce35a8f docker mount 경로 추가 2026-02-12 16:14:19 +09:00
7f3f5dca40 Merge pull request 'feat/training_260202' (#51) from feat/training_260202 into develop
Reviewed-on: #51
2026-02-12 16:13:19 +09:00
4a0a4e35ed 학습 실행 수정 2026-02-12 16:12:58 +09:00
ae055dca1e 모델등록 수정 2026-02-12 16:01:14 +09:00
26e8e1492f Merge pull request 'feat/training_260202' (#50) from feat/training_260202 into develop
Reviewed-on: #50
2026-02-12 15:52:09 +09:00
8fa722011c 모델등록 수정 2026-02-12 15:51:54 +09:00
17d47d6200 Merge branch 'feat/training_260202' of https://kamco.git.gs.dabeeo.com/MVPTeam/kamco-train-api into feat/training_260202 2026-02-12 15:47:10 +09:00
e178f58fe2 chunk save log 추가 2026-02-12 15:47:06 +09:00
cd0cf5726d Merge pull request 'feat/training_260202' (#49) from feat/training_260202 into develop
Reviewed-on: #49
2026-02-12 15:44:11 +09:00
8e4bea53da 모델등록 수정 2026-02-12 15:43:52 +09:00
7a22d8ba73 containerName 생성 변경 2026-02-12 15:39:12 +09:00
2df4a7a80b csv 파일 읽는 경로 읽어서 수정, train은 epoch + 1 해서 저장 2026-02-12 15:24:30 +09:00
b451f697bc 모델 마스터 테이블 request,response 경로 추가 2026-02-12 14:59:35 +09:00
7e9c867f34 Merge pull request '모델 등록할 때 step1State를 READY로 업데이트' (#48) from feat/training_260202 into develop
Reviewed-on: #48
2026-02-12 14:35:52 +09:00
130e85f8a1 모델 등록할 때 step1State를 READY로 업데이트 2026-02-12 14:35:17 +09:00
9e713cb49d Merge pull request '업로드 로직 재수정' (#47) from feat/training_260202 into develop
Reviewed-on: #47
2026-02-12 14:21:57 +09:00
51dfa97900 업로드 로직 재수정 2026-02-12 14:21:08 +09:00
87c6b599b4 Merge pull request 'feat/training_260202' (#46) from feat/training_260202 into develop
Reviewed-on: #46
2026-02-12 12:10:04 +09:00
f50855a822 Merge branch 'feat/training_260202' of https://kamco.git.gs.dabeeo.com/MVPTeam/kamco-train-api into feat/training_260202 2026-02-12 12:08:04 +09:00
8d416317a8 베스트 에폭 API, 2단계 실행 시 best epoch 업데이트 2026-02-12 12:07:44 +09:00
22aa071476 Merge pull request 'feat/training_260202' (#45) from feat/training_260202 into develop
Reviewed-on: #45
2026-02-12 12:06:04 +09:00
a83bd09f8f containerName 생성 변경 2026-02-12 12:05:30 +09:00
96035f864a containerName 생성 변경 2026-02-12 11:42:38 +09:00
fd7dfd7e7f containerName 생성 변경 2026-02-12 11:10:28 +09:00
190b93bee8 실행 오류 수정 2026-02-12 10:58:51 +09:00
c5f19cc961 실행 오류 수정 2026-02-12 10:58:32 +09:00
c56c0ca605 실행 오류 수정 2026-02-12 10:58:26 +09:00
c6e721aa37 실행 오류 수정 2026-02-12 10:58:12 +09:00
6572e17f00 실행 오류 수정 2026-02-12 10:51:15 +09:00
be6365807c Merge pull request '실행 오류 수정' (#43) from feat/training_260202 into develop
Reviewed-on: #43
2026-02-12 10:20:05 +09:00
d2fff7dfde 실행 오류 수정 2026-02-12 10:19:44 +09:00
f66bc22c95 Merge pull request '실행 오류 수정' (#42) from feat/training_260202 into develop
Reviewed-on: #42
2026-02-12 10:14:54 +09:00
3367d0e7be 실행 오류 수정 2026-02-12 10:14:32 +09:00
352ec6ccb0 Merge pull request 'feat/training_260202' (#41) from feat/training_260202 into develop
Reviewed-on: #41
2026-02-12 09:53:02 +09:00
6a989255a3 모델별 데이터셋 목록 - G2,G3 dataTypeName 추가 2026-02-12 09:52:24 +09:00
878b21573f 테스트 실행 추가 2026-02-11 22:00:35 +09:00
0602db1436 Merge pull request '테스트 실행 추가' (#40) from feat/training_260202 into develop
Reviewed-on: #40
2026-02-11 21:58:58 +09:00
2f8bd1f98c 테스트 실행 추가 2026-02-11 21:58:25 +09:00
75231ccbba Merge pull request '추론 실행 추가' (#39) from feat/training_260202 into develop
Reviewed-on: #39
2026-02-11 20:22:01 +09:00
1249a80da5 추론 실행 추가 2026-02-11 20:21:25 +09:00
00c78eb42f Merge pull request '성능정보 그래프 데이터 API 추가' (#38) from feat/training_260202 into develop
Reviewed-on: #38
2026-02-11 19:52:23 +09:00
35767adba1 성능정보 그래프 데이터 API 추가 2026-02-11 19:52:00 +09:00
47a2a159ef Merge pull request 'test metrics 스케줄 추가' (#37) from feat/training_260202 into develop
Reviewed-on: #37
2026-02-11 19:10:37 +09:00
95548223cd test metrics 스케줄 추가 2026-02-11 19:09:58 +09:00
2debdc5312 Merge pull request 'feat/training_260202' (#36) from feat/training_260202 into develop
Reviewed-on: #36
2026-02-11 18:51:01 +09:00
207cc47f1b 스케줄 주석 2026-02-11 18:50:43 +09:00
b6338bce8e 테이블 구조 변경 2026-02-11 18:49:59 +09:00
2cfa2adcf5 tb_model_master 컬럼 추가 2026-02-11 17:21:48 +09:00
d7e19abfc9 uploadRate 로직 수정 2026-02-11 17:06:02 +09:00
c843703ee7 Merge pull request 'file 가져오기 86 호출하는 거로 추가' (#35) from feat/training_260202 into develop
Reviewed-on: #35
2026-02-11 16:53:25 +09:00
133ea6b1ba file 가져오기 86 호출하는 거로 추가 2026-02-11 16:49:48 +09:00
0df977ae81 Merge pull request '업로드 로직 86으로 수행하기 수정' (#34) from feat/training_260202 into develop
Reviewed-on: #34
2026-02-11 16:33:03 +09:00
3e39006822 업로드 로직 86으로 수행하기 수정 2026-02-11 16:32:40 +09:00
3ec1a71406 Merge pull request '업로드 로직 수정' (#33) from feat/training_260202 into develop
Reviewed-on: #33
2026-02-11 15:53:21 +09:00
16009f1623 업로드 로직 수정 2026-02-11 15:52:57 +09:00
41911014c9 Merge pull request '업로드 로직 수정' (#32) from feat/training_260202 into develop
Reviewed-on: #32
2026-02-11 15:44:54 +09:00
8ea32ce675 업로드 로직 수정 2026-02-11 15:44:18 +09:00
a4ac80c787 Merge pull request '업로드 경로 수정' (#31) from feat/training_260202 into develop
Reviewed-on: #31
2026-02-11 15:11:02 +09:00
3a5d136d34 업로드 경로 수정 2026-02-11 15:10:37 +09:00
2f63b9ddcd Merge pull request 'feat/training_260202' (#30) from feat/training_260202 into develop
Reviewed-on: #30
2026-02-11 14:08:58 +09:00
92de48b55e 전이학습 상세 로직 수정 2026-02-11 14:08:21 +09:00
224ddae68b 전이학습 상세 수정 2026-02-11 14:05:15 +09:00
885b72a0c6 Merge pull request '모델별 데이터셋 목록 조회 수정' (#29) from feat/training_260202 into develop
Reviewed-on: #29
2026-02-11 12:29:08 +09:00
9ac00d37c5 모델별 데이터셋 목록 조회 수정 2026-02-11 12:28:38 +09:00
fbb5a34867 Merge pull request '업로드 경로 원복' (#28) from feat/training_260202 into develop
Reviewed-on: #28
2026-02-11 12:12:43 +09:00
e25fc01b25 업로드 경로 원복 2026-02-11 12:12:08 +09:00
6b3f22dd66 Merge pull request '업로드 파일 max 수정' (#27) from feat/training_260202 into develop
Reviewed-on: #27
2026-02-11 11:53:04 +09:00
abc2c8e806 업로드 파일 max 수정 2026-02-11 11:52:41 +09:00
118 changed files with 9583 additions and 361 deletions

6
.gitignore vendored
View File

@@ -72,3 +72,9 @@ docker-compose.override.yml
*.swo *.swo
*~ *~
!/CLAUDE.md !/CLAUDE.md
### SSL Certificates ###
nginx/ssl/
*.crt
*.key
*.pem

415
DEPLOY.md Normal file
View File

@@ -0,0 +1,415 @@
# KAMCO Training API 배포 가이드 (RedHat 9.6)
## 빠른 배포 (Quick Start)
이 문서는 RedHat 9.6 환경에서 HTTPS로 KAMCO Training API를 배포하는 방법을 설명합니다.
**접속 URL**:
- `https://api.train-kamco.com`
- `https://train-kamco.com`
## 사전 요구사항
- [x] Docker & Docker Compose 설치
- [x] Git 설치
- [x] sudo 권한
- [x] 포트 80, 443 사용 가능
## 1단계: /etc/hosts 설정
```bash
# root 권한으로 도메인 추가
echo "127.0.0.1 api.train-kamco.com train-kamco.com" | sudo tee -a /etc/hosts
# 확인
cat /etc/hosts | grep train-kamco
```
**예상 결과**:
```
127.0.0.1 api.train-kamco.com train-kamco.com
```
## 2단계: 방화벽 설정 (필요시)
```bash
# 방화벽 상태 확인
sudo firewall-cmd --state
# HTTP/HTTPS 포트 개방
sudo firewall-cmd --permanent --add-port=80/tcp
sudo firewall-cmd --permanent --add-port=443/tcp
# 방화벽 재로드
sudo firewall-cmd --reload
# 확인
sudo firewall-cmd --list-ports
```
**예상 결과**: `80/tcp 443/tcp`
## 3단계: 프로젝트 디렉토리로 이동
```bash
cd /path/to/kamco-train-api
# 현재 위치 확인
pwd
# 예상: /home/username/kamco-train-api
```
## 4단계: 파일 구조 확인
배포 전 필수 파일이 모두 있는지 확인하세요:
```bash
# SSL 인증서 확인
ls -la nginx/ssl/
# 예상 결과:
# train-kamco.com.crt (인증서)
# train-kamco.com.key (개인 키)
# openssl.cnf (설정 파일)
```
```bash
# Docker Compose 파일 확인
ls -la docker-compose-prod.yml nginx/nginx.conf
# 예상: 두 파일 모두 존재
```
## 5단계: Docker 네트워크 생성 (최초 1회)
```bash
# kamco-cds 네트워크가 있는지 확인
docker network ls | grep kamco-cds
# 없으면 생성
docker network create kamco-cds
```
## 6단계: Docker Compose 배포
```bash
# 기존 컨테이너 중지 (있는 경우)
docker-compose -f docker-compose-prod.yml down
# 새로운 이미지 빌드 및 실행
docker-compose -f docker-compose-prod.yml up -d --build
# 컨테이너 상태 확인
docker-compose -f docker-compose-prod.yml ps
```
**예상 결과**:
```
NAME STATUS
kamco-cd-nginx Up (healthy)
kamco-cd-training-api Up (healthy)
```
## 7단계: 배포 확인
### 컨테이너 로그 확인
```bash
# Nginx 로그
docker logs kamco-cd-nginx --tail 50
# API 로그
docker logs kamco-cd-training-api --tail 50
# 실시간 로그 (Ctrl+C로 종료)
docker-compose -f docker-compose-prod.yml logs -f
```
### HTTP → HTTPS 리다이렉트 테스트
```bash
# HTTP 접속 시 HTTPS로 리다이렉트되는지 확인
curl -I http://api.train-kamco.com
curl -I http://train-kamco.com
# 예상 결과: 301 Moved Permanently
# Location: https://api.train-kamco.com/ 또는 https://train-kamco.com/
```
### HTTPS 헬스체크
```bash
# -k 플래그: 사설 인증서 경고 무시
curl -k https://api.train-kamco.com/monitor/health
curl -k https://train-kamco.com/monitor/health
# 예상 결과: {"status":"UP","components":{...}}
```
### 브라우저 테스트
브라우저에서 다음 URL에 접속:
- `https://api.train-kamco.com/monitor/health`
- `https://train-kamco.com/monitor/health`
**사설 인증서 경고**:
- "안전하지 않음" 경고가 표시되면 **"고급"** → **"계속 진행"** 클릭
## 8단계: SSL 인증서 확인 (선택사항)
```bash
# 인증서 정보 확인
openssl x509 -in nginx/ssl/train-kamco.com.crt -text -noout | head -30
# 유효 기간 확인 (100년)
openssl x509 -in nginx/ssl/train-kamco.com.crt -noout -dates
# SAN (멀티 도메인) 확인
openssl x509 -in nginx/ssl/train-kamco.com.crt -text -noout | grep -A1 "Subject Alternative Name"
# 예상 결과:
# X509v3 Subject Alternative Name:
# DNS:api.train-kamco.com, DNS:train-kamco.com
```
## 트러블슈팅
### 문제 1: "Connection refused"
**원인**: 컨테이너가 실행되지 않음
**해결**:
```bash
# 컨테이너 상태 확인
docker ps -a | grep kamco-cd
# 컨테이너 재시작
docker-compose -f docker-compose-prod.yml restart
# 로그 확인
docker logs kamco-cd-nginx
docker logs kamco-cd-training-api
```
### 문제 2: "502 Bad Gateway"
**원인**: Nginx는 실행 중이지만 API 컨테이너가 준비되지 않음
**해결**:
```bash
# API 컨테이너 상태 확인
docker logs kamco-cd-training-api
# API 헬스체크 (컨테이너 내부에서)
docker exec kamco-cd-nginx wget -qO- http://kamco-changedetection-api:8080/monitor/health
# API 컨테이너 재시작
docker-compose -f docker-compose-prod.yml restart kamco-changedetection-api
```
### 문제 3: "Name or service not known"
**원인**: /etc/hosts에 도메인이 설정되지 않음
**해결**:
```bash
# /etc/hosts 확인
cat /etc/hosts | grep train-kamco
# 없으면 추가
echo "127.0.0.1 api.train-kamco.com train-kamco.com" | sudo tee -a /etc/hosts
```
### 문제 4: 포트 80 또는 443이 이미 사용 중
**원인**: 다른 프로세스가 포트를 사용 중
**해결**:
```bash
# 포트 사용 확인
sudo lsof -i :80
sudo lsof -i :443
# 사용 중인 프로세스 종료 (예: httpd, nginx)
sudo systemctl stop httpd
sudo systemctl stop nginx
# Docker Compose 재시작
docker-compose -f docker-compose-prod.yml restart
```
### 문제 5: SELinux 권한 오류
**원인**: SELinux가 Docker 볼륨 마운트를 차단
**해결**:
```bash
# SELinux 상태 확인
getenforce
# Permissive 모드로 임시 변경 (재부팅 시 초기화됨)
sudo setenforce 0
# 영구 변경 (권장하지 않음)
sudo vi /etc/selinux/config
# SELINUX=permissive 또는 SELINUX=disabled로 변경
```
## 컨테이너 관리 명령어
### 시작/중지/재시작
```bash
# 시작
docker-compose -f docker-compose-prod.yml up -d
# 중지
docker-compose -f docker-compose-prod.yml down
# 재시작
docker-compose -f docker-compose-prod.yml restart
# 특정 서비스만 재시작
docker-compose -f docker-compose-prod.yml restart nginx
docker-compose -f docker-compose-prod.yml restart kamco-changedetection-api
```
### 로그 확인
```bash
# 전체 로그
docker-compose -f docker-compose-prod.yml logs
# 특정 서비스 로그
docker-compose -f docker-compose-prod.yml logs nginx
docker-compose -f docker-compose-prod.yml logs kamco-changedetection-api
# 실시간 로그
docker-compose -f docker-compose-prod.yml logs -f
# 마지막 N줄만 보기
docker logs kamco-cd-nginx --tail 100
```
### 컨테이너 상태 확인
```bash
# 실행 중인 컨테이너
docker-compose -f docker-compose-prod.yml ps
# 상세 정보
docker inspect kamco-cd-nginx
docker inspect kamco-cd-training-api
# 리소스 사용량
docker stats kamco-cd-nginx kamco-cd-training-api
```
### 컨테이너 내부 접속
```bash
# Nginx 컨테이너 내부 접속
docker exec -it kamco-cd-nginx sh
# API 컨테이너 내부 접속
docker exec -it kamco-cd-training-api sh
# 내부에서 빠져나오기
exit
```
## 업데이트 및 재배포
### 코드 업데이트 후 재배포
```bash
# 1. Git pull (코드 업데이트)
git pull origin develop
# 2. JAR 파일 빌드 (Jenkins에서 수행하는 경우 생략)
./gradlew clean build -x test
# 3. 컨테이너 재빌드 및 재시작
docker-compose -f docker-compose-prod.yml down
docker-compose -f docker-compose-prod.yml up -d --build
# 4. 로그 확인
docker-compose -f docker-compose-prod.yml logs -f
```
### 설정 파일만 변경한 경우
```bash
# nginx.conf 또는 docker-compose-prod.yml 변경 시
docker-compose -f docker-compose-prod.yml down
docker-compose -f docker-compose-prod.yml up -d
# 또는
docker-compose -f docker-compose-prod.yml restart nginx
```
## 모니터링
### 헬스체크 엔드포인트
```bash
# API 헬스체크
curl -k https://api.train-kamco.com/monitor/health
# 예상 결과:
# {
# "status": "UP",
# "components": {
# "db": {"status": "UP"},
# "diskSpace": {"status": "UP"}
# }
# }
```
### 시스템 리소스
```bash
# 디스크 사용량
df -h
# 메모리 사용량
free -h
# Docker 이미지 및 컨테이너 용량
docker system df
```
## 보안 권장 사항
1. **사설 인증서**: 현재 사설 인증서를 사용 중입니다. 프로덕션 환경에서는 **Let's Encrypt** 또는 **GlobalSign** 같은 공인 인증서 사용을 권장합니다.
2. **방화벽**: 필요한 포트(80, 443)만 개방하고, 불필요한 포트는 차단하세요.
3. **정기 업데이트**: Docker 이미지와 시스템 패키지를 정기적으로 업데이트하세요.
4. **로그 모니터링**: 정기적으로 로그를 확인하여 비정상적인 활동을 감지하세요.
5. **백업**: SSL 인증서 키 파일(`train-kamco.com.key`)과 데이터베이스를 정기적으로 백업하세요.
## 참고 문서
- **SSL 인증서 설정**: [nginx/SSL_SETUP.md](nginx/SSL_SETUP.md)
- **프로젝트 개요**: [README.md](README.md)
- **CLAUDE.md**: [CLAUDE.md](CLAUDE.md)
## 지원
문제가 발생하면 다음을 확인하세요:
1. 컨테이너 로그: `docker-compose -f docker-compose-prod.yml logs`
2. 컨테이너 상태: `docker-compose -f docker-compose-prod.yml ps`
3. /etc/hosts 설정: `cat /etc/hosts | grep train-kamco`
4. 방화벽 상태: `sudo firewall-cmd --list-ports`
---
**배포 완료!** 🎉
접속 URL:
- `https://api.train-kamco.com`
- `https://train-kamco.com`

443
DEPLOYMENT.md Normal file
View File

@@ -0,0 +1,443 @@
# KAMCO Train API - Production Deployment Guide
프로덕션 환경 배포 가이드
## 목차
- [사전 요구사항](#사전-요구사항)
- [초기 설정](#초기-설정)
- [배포 순서](#배포-순서)
- [개별 서비스 관리](#개별-서비스-관리)
- [롤백 절차](#롤백-절차)
- [모니터링 및 헬스체크](#모니터링-및-헬스체크)
- [트러블슈팅](#트러블슈팅)
---
## 사전 요구사항
### 시스템 요구사항
- Docker Engine 20.10+
- Docker Compose 2.0+
- 최소 메모리: 4GB
- 디스크 공간: 20GB 이상
### 네트워크 요구사항
- 도메인 설정:
- `train-kamco.com` → 서버 IP (Web UI)
- `api.train-kamco.com` → 서버 IP (API)
- 포트:
- 80 (HTTP)
- 443 (HTTPS)
- 8080 (API - internal)
- 3002 (Web - internal)
### 필수 파일
- SSL 인증서:
- `nginx/ssl/train-kamco.com.crt`
- `nginx/ssl/train-kamco.com.key`
- 환경 설정:
- `application-prod.yml` (API 설정)
- `.env` 파일 (IMAGE_TAG 등)
---
## 초기 설정
### 1. Docker 네트워크 생성
```bash
# kamco-cds 네트워크 생성 (최초 1회만)
docker network create kamco-cds
# 네트워크 확인
docker network ls | grep kamco-cds
```
### 2. SSL 인증서 배치
```bash
# 인증서 디렉토리 생성
mkdir -p nginx/ssl
# 인증서 파일 복사
cp /path/to/train-kamco.com.crt nginx/ssl/
cp /path/to/train-kamco.com.key nginx/ssl/
# 권한 설정
chmod 600 nginx/ssl/train-kamco.com.key
chmod 644 nginx/ssl/train-kamco.com.crt
```
### 3. 환경 변수 설정
```bash
# .env 파일 생성
cat > .env << EOF
IMAGE_TAG=latest
SPRING_PROFILES_ACTIVE=prod
TZ=Asia/Seoul
EOF
```
### 4. 볼륨 디렉토리 생성
```bash
# 데이터 디렉토리 생성
mkdir -p ./app/model_output
mkdir -p ./app/train_dataset
# 권한 설정
chmod -R 755 ./app
```
---
## 배포 순서
### 전체 스택 초기 배포
**중요**: 반드시 아래 순서대로 실행해야 합니다.
```bash
# 1. Nginx 시작
docker-compose -f docker-compose-nginx.yml up -d
# 2. Nginx 상태 확인
docker ps | grep kamco-train-nginx
# 3. API 서비스 시작
docker-compose -f docker-compose-prod.yml up -d
# 4. API 헬스체크 대기 (최대 40초)
sleep 40
curl -f http://localhost:8080/monitor/health
# 5. Web 서비스 시작 (kamco-train-web 프로젝트에서)
# cd ../kamco-train-web
# docker-compose -f docker-compose-prod.yml up -d
# 6. 전체 상태 확인
docker ps -a | grep kamco
```
### 배포 검증
```bash
# 서비스별 헬스체크
curl -f http://localhost:8080/monitor/health # API (internal)
curl -kf https://api.train-kamco.com/monitor/health # API (external)
curl -kf https://train-kamco.com # Web (external)
# Nginx 설정 검증
docker exec kamco-train-nginx nginx -t
# 로그 확인
docker-compose -f docker-compose-nginx.yml logs --tail=50
docker-compose -f docker-compose-prod.yml logs --tail=50
```
---
## 개별 서비스 관리
### Nginx 관리
```bash
# 설정 변경 후 리로드 (다운타임 없음)
docker exec kamco-train-nginx nginx -s reload
# 재시작
docker-compose -f docker-compose-nginx.yml restart
# 로그 확인
docker-compose -f docker-compose-nginx.yml logs -f
# 컨테이너 내부 접근
docker exec -it kamco-train-nginx sh
```
### API 서비스 관리
```bash
# 재배포 (새 이미지 빌드)
docker-compose -f docker-compose-prod.yml up -d --build
# 재시작 (이미지 변경 없이)
docker-compose -f docker-compose-prod.yml restart
# 중지
docker-compose -f docker-compose-prod.yml down
# 로그 확인
docker-compose -f docker-compose-prod.yml logs -f kamco-train-api
# 컨테이너 내부 접근
docker exec -it kamco-train-api bash
```
### Web 서비스 관리
```bash
# kamco-train-web 프로젝트에서 실행
cd ../kamco-train-web
# 재배포
docker-compose -f docker-compose-prod.yml up -d --build
# 재시작
docker-compose -f docker-compose-prod.yml restart
# 로그 확인
docker-compose -f docker-compose-prod.yml logs -f
```
---
## 롤백 절차
### 이미지 기반 롤백
```bash
# 1. 사용 가능한 이미지 확인
docker images | grep kamco-train-api
# 2. 이전 이미지 태그로 롤백
export IMAGE_TAG=previous-commit-hash
docker-compose -f docker-compose-prod.yml up -d
# 3. 헬스체크 확인
curl -f http://localhost:8080/monitor/health
```
### Git 기반 롤백
```bash
# 1. 이전 커밋으로 체크아웃
git log --oneline -10
git checkout <previous-commit-hash>
# 2. 재빌드 및 배포
docker-compose -f docker-compose-prod.yml up -d --build
# 3. 검증 후 브랜치 업데이트 (필요시)
# git checkout develop
# git reset --hard <previous-commit-hash>
# git push -f origin develop
```
---
## 모니터링 및 헬스체크
### 헬스체크 엔드포인트
```bash
# API 헬스체크
curl http://localhost:8080/monitor/health
curl http://localhost:8080/monitor/health/readiness
curl http://localhost:8080/monitor/health/liveness
# Nginx를 통한 헬스체크
curl -k https://api.train-kamco.com/monitor/health
```
### 컨테이너 상태 모니터링
```bash
# 모든 컨테이너 상태
docker ps -a | grep kamco
# 리소스 사용량 실시간 모니터링
docker stats kamco-train-nginx kamco-train-api
# 헬스체크 상태
docker inspect kamco-train-api | grep -A 10 Health
```
### 로그 모니터링
```bash
# 실시간 로그 (모든 서비스)
docker-compose -f docker-compose-nginx.yml logs -f &
docker-compose -f docker-compose-prod.yml logs -f &
# 에러 로그만 필터링
docker-compose -f docker-compose-prod.yml logs | grep -i error
# 최근 100줄
docker-compose -f docker-compose-prod.yml logs --tail=100
```
---
## 트러블슈팅
### 1. Nginx 502 Bad Gateway
**원인**: API 서비스가 준비되지 않음
```bash
# API 컨테이너 상태 확인
docker ps | grep kamco-train-api
# API 로그 확인
docker logs kamco-train-api --tail=100
# 네트워크 연결 확인
docker network inspect kamco-cds | grep kamco-train-api
# 해결: API 재시작
docker-compose -f docker-compose-prod.yml restart
```
### 2. SSL 인증서 오류
**원인**: 인증서 파일 누락 또는 권한 문제
```bash
# 인증서 파일 확인
ls -la nginx/ssl/
# Nginx 설정 검증
docker exec kamco-train-nginx nginx -t
# 해결: 인증서 재배치 및 권한 설정
chmod 600 nginx/ssl/train-kamco.com.key
chmod 644 nginx/ssl/train-kamco.com.crt
docker-compose -f docker-compose-nginx.yml restart
```
### 3. 컨테이너 시작 실패
**원인**: 포트 충돌, 볼륨 권한, 메모리 부족
```bash
# 포트 사용 확인
netstat -tulpn | grep -E '80|443|8080'
# 볼륨 권한 확인
ls -la ./app/
# 메모리 사용량 확인
free -h
docker system df
# 해결: 충돌 프로세스 종료 또는 포트 변경
# 메모리 정리
docker system prune -a
```
### 4. 네트워크 연결 문제
**원인**: kamco-cds 네트워크 미생성 또는 컨테이너 미연결
```bash
# 네트워크 확인
docker network ls | grep kamco-cds
# 네트워크 상세 정보
docker network inspect kamco-cds
# 해결: 네트워크 생성
docker network create kamco-cds
# 컨테이너를 네트워크에 연결
docker network connect kamco-cds kamco-train-api
docker network connect kamco-cds kamco-train-nginx
```
### 5. 데이터베이스 연결 실패
**원인**: application-prod.yml의 DB 설정 오류
```bash
# API 로그에서 DB 연결 에러 확인
docker logs kamco-train-api | grep -i "connection"
# DB 호스트 연결 테스트
docker exec kamco-train-api ping <db-host>
# 해결: application-prod.yml 수정 후 재배포
vim src/main/resources/application-prod.yml
docker-compose -f docker-compose-prod.yml up -d --build
```
---
## Jenkins CI/CD 연동
현재 프로젝트는 Jenkins 파이프라인으로 자동 배포됩니다.
### Jenkinsfile-dev 주요 단계
1. **Checkout**: develop 브랜치 체크아웃
2. **Build**: `./gradlew clean build -x test`
3. **Extract Commit**: IMAGE_TAG로 사용
4. **Transfer**: 배포 서버로 파일 전송
5. **Deploy**: Docker Compose 빌드 및 배포
6. **Health Check**: 30초 대기 후 헬스체크
7. **Cleanup**: 오래된 이미지 정리 (최신 5개 유지)
### 배포 서버 정보
- **서버**: 192.168.2.109
- **사용자**: space
- **배포 경로**: `/home/space/kamco-training-api`
- **헬스체크**: `http://localhost:7200/monitor/health`
---
## 백업 및 복구
### 데이터 백업
```bash
# 볼륨 데이터 백업
tar -czf backup-$(date +%Y%m%d).tar.gz ./app/model_output ./app/train_dataset
# 설정 파일 백업
tar -czf config-backup-$(date +%Y%m%d).tar.gz \
nginx/nginx.conf \
nginx/ssl/ \
src/main/resources/application-prod.yml
```
### 이미지 백업
```bash
# 현재 이미지 저장
docker save kamco-train-api:latest | gzip > kamco-train-api-latest.tar.gz
# 이미지 복구
gunzip -c kamco-train-api-latest.tar.gz | docker load
```
---
## 보안 체크리스트
- [ ] SSL 인증서 유효기간 확인
- [ ] nginx/ssl/ 디렉토리 권한 600
- [ ] application-prod.yml에 DB 비밀번호 암호화
- [ ] JWT secret key 환경변수로 관리
- [ ] Docker 소켓 권한 최소화
- [ ] 방화벽 규칙 설정 (80, 443만 외부 노출)
- [ ] 정기 보안 업데이트 (docker images)
---
## 참고 문서
- [CLAUDE.md](./CLAUDE.md) - 프로젝트 개발 가이드
- [README.md](./README.md) - 프로젝트 개요
- [Jenkinsfile-dev](./Jenkinsfile-dev) - CI/CD 파이프라인
- [nginx/nginx.conf](./nginx/nginx.conf) - Nginx 설정
---
## 연락처
문제 발생 시:
1. 로그 수집: `docker-compose logs` 출력
2. 시스템 정보: `docker ps -a`, `docker network ls`
3. 이슈 리포트: GitHub Issues 또는 내부 이슈 트래커

20
Dockerfile Normal file
View File

@@ -0,0 +1,20 @@
# Stage 1: Build stage (gradle build는 Jenkins에서 이미 수행)
FROM eclipse-temurin:21-jre-jammy
# docker CLI 설치 (컨테이너에서 호스트 Docker 제어용) 260212 추가
RUN apt-get update && \
apt-get install -y --no-install-recommends docker.io ca-certificates && \
rm -rf /var/lib/apt/lists/*
# 작업 디렉토리 설정
WORKDIR /app
# JAR 파일 복사 (Jenkins에서 빌드된 ROOT.jar)
COPY build/libs/ROOT.jar app.jar
# 포트 노출
EXPOSE 8080
# 애플리케이션 실행
# dev 프로파일로 실행
ENTRYPOINT ["java", "-jar", "-Dspring.profiles.active=prod", "app.jar"]

View File

@@ -1,6 +1,11 @@
# Stage 1: Build stage (gradle build는 Jenkins에서 이미 수행) # Stage 1: Build stage (gradle build는 Jenkins에서 이미 수행)
FROM eclipse-temurin:21-jre-jammy FROM eclipse-temurin:21-jre-jammy
# docker CLI 설치 (컨테이너에서 호스트 Docker 제어용) 260212 추가
RUN apt-get update && \
apt-get install -y --no-install-recommends docker.io ca-certificates && \
rm -rf /var/lib/apt/lists/*
# 작업 디렉토리 설정 # 작업 디렉토리 설정
WORKDIR /app WORKDIR /app

View File

@@ -3,6 +3,7 @@ plugins {
id 'org.springframework.boot' version '3.5.7' id 'org.springframework.boot' version '3.5.7'
id 'io.spring.dependency-management' version '1.1.7' id 'io.spring.dependency-management' version '1.1.7'
id 'com.diffplug.spotless' version '6.25.0' id 'com.diffplug.spotless' version '6.25.0'
id 'idea'
} }
group = 'com.kamco.cd' group = 'com.kamco.cd'
@@ -21,11 +22,23 @@ configurations {
} }
} }
// QueryDSL 생성된 소스 디렉토리 정의
def generatedSourcesDir = file("$buildDir/generated/sources/annotationProcessor/java/main")
repositories { repositories {
mavenCentral() mavenCentral()
maven { url "https://repo.osgeo.org/repository/release/" } maven { url "https://repo.osgeo.org/repository/release/" }
} }
// Gradle이 생성된 소스를 컴파일 경로에 포함하도록 설정
sourceSets {
main {
java {
srcDirs += generatedSourcesDir
}
}
}
dependencies { dependencies {
implementation 'org.springframework.boot:spring-boot-starter-data-jpa' implementation 'org.springframework.boot:spring-boot-starter-data-jpa'
implementation 'org.springframework.boot:spring-boot-starter-web' implementation 'org.springframework.boot:spring-boot-starter-web'
@@ -83,6 +96,23 @@ dependencies {
implementation 'io.hypersistence:hypersistence-utils-hibernate-63:3.7.0' implementation 'io.hypersistence:hypersistence-utils-hibernate-63:3.7.0'
implementation 'org.reflections:reflections:0.10.2' implementation 'org.reflections:reflections:0.10.2'
implementation 'com.jcraft:jsch:0.1.55'
implementation 'org.apache.commons:commons-csv:1.10.0'
}
// IntelliJ가 생성된 소스를 인식하도록 설정
idea {
module {
// 소스 디렉토리로 인식
sourceDirs += generatedSourcesDir
// Generated Sources Root로 마킹 (IntelliJ에서 특별 처리)
generatedSourceDirs += generatedSourcesDir
// 소스 및 Javadoc 다운로드
downloadJavadoc = true
downloadSources = true
}
} }
configurations.configureEach { configurations.configureEach {
@@ -93,6 +123,21 @@ tasks.named('test') {
useJUnitPlatform() useJUnitPlatform()
} }
// 컴파일 전 생성된 소스 디렉토리 생성 보장
tasks.named('compileJava') {
doFirst {
generatedSourcesDir.mkdirs()
}
}
// 생성된 소스 정리 태스크
tasks.register('cleanGeneratedSources', Delete) {
delete generatedSourcesDir
}
tasks.named('clean') {
dependsOn 'cleanGeneratedSources'
}
bootJar { bootJar {
archiveFileName = 'ROOT.jar' archiveFileName = 'ROOT.jar'

View File

@@ -5,6 +5,13 @@ services:
dockerfile: Dockerfile-dev dockerfile: Dockerfile-dev
image: kamco-cd-training-api:${IMAGE_TAG:-latest} image: kamco-cd-training-api:${IMAGE_TAG:-latest}
container_name: kamco-cd-training-api container_name: kamco-cd-training-api
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
ports: ports:
- "7200:8080" - "7200:8080"
environment: environment:
@@ -14,6 +21,8 @@ services:
- /mnt/nfs_share/images:/app/original-images - /mnt/nfs_share/images:/app/original-images
- /mnt/nfs_share/model_output:/app/model-outputs - /mnt/nfs_share/model_output:/app/model-outputs
- /mnt/nfs_share/train_dataset:/app/train-dataset - /mnt/nfs_share/train_dataset:/app/train-dataset
- /home/kcomu/data:/home/kcomu/data
- /var/run/docker.sock:/var/run/docker.sock
networks: networks:
- kamco-cds - kamco-cds
restart: unless-stopped restart: unless-stopped

28
docker-compose-nginx.yml Normal file
View File

@@ -0,0 +1,28 @@
services:
nginx:
image: nginx:alpine
container_name: kamco-train-nginx
ports:
- "80:80"
- "443:443"
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
- ./nginx/ssl:/etc/nginx/ssl:ro
- nginx-logs:/var/log/nginx
networks:
- kamco-cds
restart: unless-stopped
healthcheck:
test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "--no-check-certificate", "https://localhost/monitor/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 10s
networks:
kamco-cds:
external: true
volumes:
nginx-logs:
driver: local

36
docker-compose-prod.yml Normal file
View File

@@ -0,0 +1,36 @@
services:
kamco-train-api:
build:
context: .
dockerfile: Dockerfile
image: kamco-train-api:${IMAGE_TAG:-latest}
container_name: kamco-train-api
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
expose:
- "8080"
environment:
- SPRING_PROFILES_ACTIVE=prod
- TZ=Asia/Seoul
volumes:
- ./app/model_output:/app/model-outputs
- ./app/train_dataset:/app/train-dataset
- /var/run/docker.sock:/var/run/docker.sock
networks:
- kamco-cds
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/monitor/health"]
interval: 10s
timeout: 5s
retries: 5
start_period: 40s
networks:
kamco-cds:
external: true

414
nginx/SSL_SETUP.md Normal file
View File

@@ -0,0 +1,414 @@
# SSL 사설 인증서 설정 가이드 (RedHat 9.6)
## 개요
이 문서는 RedHat 9.6 환경에서 `https://api.train-kamco.com``https://train-kamco.com` 도메인을 위한 100년 유효한 사설 SSL 인증서 설정 방법을 설명합니다.
## 디렉토리 구조
```
nginx/
├── nginx.conf # Nginx 설정 파일
├── ssl/
│ ├── openssl.cnf # OpenSSL 설정 파일 (SAN 포함)
│ ├── train-kamco.com.crt # 사설 SSL 인증서 (멀티 도메인)
│ └── train-kamco.com.key # 개인 키 (비공개)
└── SSL_SETUP.md # 이 문서
```
## 인증서 정보
- **도메인**: api.train-kamco.com, train-kamco.com (멀티 도메인)
- **유효 기간**: 100년 (36500일)
- **알고리즘**: RSA 4096-bit
- **CN (Common Name)**: api.train-kamco.com
- **SAN (Subject Alternative Names)**: api.train-kamco.com, train-kamco.com
## 사설 SSL 인증서 생성 (이미 생성됨)
인증서가 이미 생성되어 있습니다. 재생성이 필요한 경우 아래 단계를 따르세요.
### 1. OpenSSL 설정 파일 생성
```bash
cd /path/to/kamco-train-api
cat > nginx/ssl/openssl.cnf << 'EOF'
[req]
default_bits = 4096
prompt = no
default_md = sha256
distinguished_name = dn
req_extensions = v3_req
[dn]
C=KR
ST=Seoul
L=Seoul
O=KAMCO
OU=Training
CN=api.train-kamco.com
[v3_req]
subjectAltName = @alt_names
[alt_names]
DNS.1 = api.train-kamco.com
DNS.2 = train-kamco.com
EOF
```
### 2. SSL 인증서 및 개인 키 생성
```bash
# nginx/ssl 디렉토리 생성 (없는 경우)
mkdir -p nginx/ssl
chmod 700 nginx/ssl
# 인증서 및 개인 키 생성 (100년 유효)
openssl req -new -x509 -newkey rsa:4096 -sha256 -nodes \
-keyout nginx/ssl/train-kamco.com.key \
-out nginx/ssl/train-kamco.com.crt \
-days 36500 \
-config nginx/ssl/openssl.cnf \
-extensions v3_req
# 파일 권한 설정
chmod 600 nginx/ssl/train-kamco.com.key
chmod 644 nginx/ssl/train-kamco.com.crt
```
### 3. 인증서 검증
```bash
# 인증서 정보 확인
openssl x509 -in nginx/ssl/train-kamco.com.crt -text -noout
# 유효 기간 확인
openssl x509 -in nginx/ssl/train-kamco.com.crt -text -noout | grep -A2 "Validity"
# SAN (멀티 도메인) 확인
openssl x509 -in nginx/ssl/train-kamco.com.crt -text -noout | grep -A1 "Subject Alternative Name"
# CN 확인
openssl x509 -in nginx/ssl/train-kamco.com.crt -noout -subject
# 개인 키 확인
openssl rsa -in nginx/ssl/train-kamco.com.key -check
```
**예상 결과**:
```
X509v3 Subject Alternative Name:
DNS:api.train-kamco.com, DNS:train-kamco.com
Validity
Not Before: Mar 2 23:39:XX 2026 GMT
Not After : Feb 6 23:39:XX 2126 GMT
```
## /etc/hosts 설정 (RedHat 9.6)
### 1. hosts 파일에 도메인 추가
```bash
# root 권한으로 실행
echo "127.0.0.1 api.train-kamco.com train-kamco.com" | sudo tee -a /etc/hosts
# 확인
cat /etc/hosts | grep train-kamco
```
**예상 결과**:
```
127.0.0.1 api.train-kamco.com train-kamco.com
```
### 2. 도메인 확인
```bash
# ping 테스트
ping -c 2 api.train-kamco.com
ping -c 2 train-kamco.com
```
## Docker Compose 배포
### 1. 기존 컨테이너 중지 (실행 중인 경우)
```bash
cd /path/to/kamco-train-api
docker-compose -f docker-compose-prod.yml down
```
### 2. Production 환경 실행
```bash
# IMAGE_TAG 환경 변수 설정 (선택사항)
export IMAGE_TAG=latest
# Docker Compose 실행
docker-compose -f docker-compose-prod.yml up -d
# 컨테이너 상태 확인
docker-compose -f docker-compose-prod.yml ps
```
### 3. 로그 확인
```bash
# Nginx 로그
docker logs kamco-cd-nginx
# API 로그
docker logs kamco-cd-training-api
# 실시간 로그 확인
docker-compose -f docker-compose-prod.yml logs -f
```
## HTTPS 접속 테스트
### 1. HTTP → HTTPS 리다이렉트 테스트
```bash
# api.train-kamco.com
curl -I http://api.train-kamco.com
# train-kamco.com
curl -I http://train-kamco.com
# 예상 결과: 301 Moved Permanently
# Location: https://api.train-kamco.com/ 또는 https://train-kamco.com/
```
### 2. HTTPS 헬스체크 (-k: 사설 인증서 경고 무시)
```bash
# api.train-kamco.com
curl -k https://api.train-kamco.com/monitor/health
# train-kamco.com
curl -k https://train-kamco.com/monitor/health
# 예상 결과: {"status":"UP","components":{...}}
```
### 3. SSL 인증서 확인
```bash
# api.train-kamco.com
openssl s_client -connect api.train-kamco.com:443 -showcerts
# train-kamco.com
openssl s_client -connect train-kamco.com:443 -showcerts
# CN 및 SAN 확인
```
### 4. 브라우저 테스트
브라우저에서 다음 URL에 접속:
- `https://api.train-kamco.com/monitor/health`
- `https://train-kamco.com/monitor/health`
**주의**: 사설 인증서이므로 "안전하지 않음" 경고가 표시됩니다.
- **Chrome/Edge**: "고급" → "계속 진행" 클릭
- **Firefox**: "위험 감수 및 계속" 클릭
## 브라우저에서 사설 인증서 신뢰 설정 (선택사항)
사설 인증서를 브라우저에 등록하면 경고 없이 접속 가능합니다.
### Chrome/Edge (RedHat Desktop)
1. `chrome://settings/certificates` 접속
2. **Authorities** 탭 선택
3. **Import** 클릭
4. `nginx/ssl/train-kamco.com.crt` 선택
5. **Trust this certificate for identifying websites** 체크
6. **OK** 클릭
### Firefox
1. `about:preferences#privacy` 접속
2. **Certificates****View Certificates** 클릭
3. **Authorities** 탭 선택
4. **Import** 클릭
5. `nginx/ssl/train-kamco.com.crt` 선택
6. **Trust this CA to identify websites** 체크
7. **OK** 클릭
## 방화벽 설정 (RedHat 9.6)
### 1. 방화벽 상태 확인
```bash
sudo firewall-cmd --state
```
### 2. HTTP (80) 및 HTTPS (443) 포트 개방
```bash
# HTTP 포트 개방
sudo firewall-cmd --permanent --add-port=80/tcp
# HTTPS 포트 개방
sudo firewall-cmd --permanent --add-port=443/tcp
# 방화벽 재로드
sudo firewall-cmd --reload
# 확인
sudo firewall-cmd --list-ports
```
**예상 결과**:
```
80/tcp 443/tcp
```
## 보안 체크리스트
- [x] `train-kamco.com.key` 파일 권한이 600으로 설정됨
- [x] ssl 디렉토리가 버전 관리에서 제외됨 (.gitignore 확인)
- [x] 두 도메인(api.train-kamco.com, train-kamco.com) 모두 SAN에 포함됨
- [ ] 방화벽에서 80, 443 포트 개방 확인
- [x] HSTS 헤더 활성화 확인
- [x] TLS 1.2 이상만 허용 확인
- [ ] /etc/hosts에 도메인 매핑 확인
## 트러블슈팅
### 인증서 관련 오류
**"certificate verify failed"**
```bash
# 해결: -k 플래그 사용 (사설 인증서 경고 무시)
curl -k https://api.train-kamco.com/monitor/health
```
**"NET::ERR_CERT_AUTHORITY_INVALID" (브라우저)**
- 정상 동작: 사설 인증서이므로 브라우저 경고는 예상된 동작입니다
- 해결: 브라우저에 인증서 등록 (위 "브라우저에서 사설 인증서 신뢰 설정" 참조)
### 연결 오류
**"Connection refused"**
```bash
# 컨테이너 상태 확인
docker ps | grep kamco-cd
# 포트 바인딩 확인
docker port kamco-cd-nginx
# 예상 결과:
# 80/tcp -> 0.0.0.0:80
# 443/tcp -> 0.0.0.0:443
```
**"502 Bad Gateway"**
```bash
# API 컨테이너 상태 확인
docker logs kamco-cd-training-api
# nginx → API 연결 확인
docker exec kamco-cd-nginx wget -qO- http://kamco-changedetection-api:8080/monitor/health
```
**"Name or service not known" (도메인 해석 실패)**
```bash
# /etc/hosts 확인
cat /etc/hosts | grep train-kamco
# 없으면 추가
echo "127.0.0.1 api.train-kamco.com train-kamco.com" | sudo tee -a /etc/hosts
```
### 방화벽 관련 오류
**외부에서 접속 안 됨**
```bash
# 방화벽 확인
sudo firewall-cmd --list-ports
# 포트 개방
sudo firewall-cmd --permanent --add-port=80/tcp
sudo firewall-cmd --permanent --add-port=443/tcp
sudo firewall-cmd --reload
```
## 인증서 만료 및 갱신
### 만료 확인
```bash
# 인증서 만료일 확인
openssl x509 -in nginx/ssl/train-kamco.com.crt -noout -enddate
# 예상 결과: notAfter=Feb 6 23:39:XX 2126 GMT (100년 후)
```
### 갱신 방법 (필요시)
100년 유효한 인증서이므로 갱신이 필요하지 않지만, 재생성이 필요한 경우:
```bash
# 기존 인증서 백업
cp nginx/ssl/train-kamco.com.crt nginx/ssl/train-kamco.com.crt.bak
cp nginx/ssl/train-kamco.com.key nginx/ssl/train-kamco.com.key.bak
# 위의 "사설 SSL 인증서 생성" 단계 재실행
# nginx 재시작
docker-compose -f docker-compose-prod.yml restart nginx
```
## 주의사항
1. **사설 인증서 경고**: 브라우저에서 "안전하지 않음" 경고가 표시됩니다. 프로덕션 환경에서는 **공인 인증서(Let's Encrypt, GlobalSign 등) 사용을 권장**합니다.
2. **포트 80/443**: Docker가 자동으로 처리하지만, 이미 사용 중인 프로세스가 있으면 충돌할 수 있습니다.
```bash
# 포트 사용 확인
sudo lsof -i :80
sudo lsof -i :443
```
3. **대용량 파일 업로드**: `client_max_body_size`를 10GB로 설정했으므로, 서버 메모리 및 디스크 용량을 충분히 확보하세요.
4. **인증서 백업**: `train-kamco.com.key` 파일은 매우 중요합니다. 안전한 곳에 백업하세요.
5. **SELinux**: RedHat 9.6에서 SELinux가 활성화된 경우, Docker 볼륨 마운트 권한 문제가 발생할 수 있습니다.
```bash
# SELinux 상태 확인
getenforce
# 필요시 permissive 모드로 변경
sudo setenforce 0
```
### 2단계: 시스템 신뢰 폴더로 복사
터미널을 열고 관리자 권한(sudo)을 사용해 인증서를 시스템 폴더로 복사합니다.
```
sudo cp mycert.crt /etc/pki/ca-trust/source/anchors/
```
### 3단계: 시스템 신뢰 목록 업데이트
아래 명령어를 입력해 추가한 인증서를 시스템에 갱신시킵니다.
```
sudo update-ca-trust
```
## 참고 자료
- [OpenSSL Documentation](https://www.openssl.org/docs/)
- [Nginx SSL Configuration](https://nginx.org/en/docs/http/configuring_https_servers.html)
- [Docker Compose Documentation](https://docs.docker.com/compose/)
- [Let's Encrypt (공인 인증서)](https://letsencrypt.org/)

179
nginx/nginx.conf Normal file
View File

@@ -0,0 +1,179 @@
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
# 로그 설정
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
error_log /var/log/nginx/error.log warn;
sendfile on;
keepalive_timeout 65;
# 업로드 파일 크기 제한 (10GB)
client_max_body_size 10G;
# Upstream 설정
upstream api_backend {
server kamco-train-api:8080;
}
upstream web_backend {
server kamco-train-web:3002;
}
# HTTP → HTTPS 리다이렉트 서버
server {
listen 80;
server_name api.train-kamco.com train-kamco.com;
# 모든 HTTP 요청을 HTTPS로 리다이렉트
return 301 https://$server_name$request_uri;
}
# HTTPS 서버 설정
server {
listen 443 ssl http2;
server_name api.train-kamco.com;
# SSL 인증서 설정 (사설 인증서 - 멀티 도메인)
ssl_certificate /etc/nginx/ssl/train-kamco.com.crt;
ssl_certificate_key /etc/nginx/ssl/train-kamco.com.key;
# SSL 프로토콜 및 암호화 설정
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384';
ssl_prefer_server_ciphers off;
# SSL 세션 캐시
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
# HSTS (HTTP Strict Transport Security)
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
# 보안 헤더
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;
# 프록시 설정
location / {
proxy_pass http://api_backend;
proxy_http_version 1.1;
# 프록시 헤더 설정
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $server_name;
# 인증 헤더 및 쿠키 전달 (JWT 토큰 전달 보장)
proxy_pass_request_headers on;
proxy_set_header Cookie $http_cookie;
proxy_set_header Authorization $http_authorization;
# 타임아웃 설정 (대용량 파일 업로드 지원)
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
# 버퍼 설정
proxy_buffering on;
proxy_buffer_size 4k;
proxy_buffers 8 4k;
proxy_busy_buffers_size 8k;
}
# 헬스체크 엔드포인트
location /monitor/health {
proxy_pass http://api_backend/monitor/health;
access_log off;
}
}
# HTTPS 서버 설정 - Next.js Web Application
server {
listen 443 ssl http2;
server_name train-kamco.com;
# SSL 인증서 설정 (사설 인증서 - 멀티 도메인)
ssl_certificate /etc/nginx/ssl/train-kamco.com.crt;
ssl_certificate_key /etc/nginx/ssl/train-kamco.com.key;
# SSL 프로토콜 및 암호화 설정
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384';
ssl_prefer_server_ciphers off;
# SSL 세션 캐시
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
# HSTS (HTTP Strict Transport Security)
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
# 보안 헤더
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;
# API 프록시 설정 (Web에서 API 호출 시)
location /api/ {
proxy_pass http://api_backend/api/;
proxy_http_version 1.1;
# 프록시 헤더 설정
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $server_name;
# 인증 헤더 및 쿠키 전달
proxy_pass_request_headers on;
proxy_set_header Cookie $http_cookie;
# 타임아웃 설정
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
}
# 프록시 설정
location / {
proxy_pass http://web_backend;
proxy_http_version 1.1;
# 프록시 헤더 설정
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $server_name;
# Next.js WebSocket 지원을 위한 Upgrade 헤더
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# 타임아웃 설정
proxy_connect_timeout 600s;
proxy_send_timeout 600s;
proxy_read_timeout 600s;
# 버퍼 설정
proxy_buffering on;
proxy_buffer_size 4k;
proxy_buffers 8 4k;
proxy_busy_buffers_size 8k;
}
}
}

View File

@@ -23,7 +23,8 @@ public class JwtAuthenticationFilter extends OncePerRequestFilter {
private final UserDetailsService userDetailsService; private final UserDetailsService userDetailsService;
private static final AntPathMatcher PATH_MATCHER = new AntPathMatcher(); private static final AntPathMatcher PATH_MATCHER = new AntPathMatcher();
private static final String[] EXCLUDE_PATHS = { private static final String[] EXCLUDE_PATHS = {
"/api/auth/signin", "/api/auth/refresh", "/api/auth/logout", "/api/members/*/password" // "/api/auth/signin", "/api/auth/refresh", "/api/auth/logout", "/api/members/*/password"
"/api/auth/signin", "/api/auth/refresh", "/api/auth/logout"
}; };
@Override @Override

View File

@@ -20,7 +20,7 @@ import org.springframework.web.bind.annotation.*;
@Tag(name = "공통코드 관리", description = "공통코드 관리 API") @Tag(name = "공통코드 관리", description = "공통코드 관리 API")
@RestController @RestController
@RequiredArgsConstructor @RequiredArgsConstructor
@RequestMapping("/api/code") @RequestMapping("/api/common-code")
public class CommonCodeApiController { public class CommonCodeApiController {
private final CommonCodeService commonCodeService; private final CommonCodeService commonCodeService;

View File

@@ -0,0 +1,48 @@
package com.kamco.cd.training.common.download;
import com.kamco.cd.training.common.download.dto.DownloadSpec;
import com.kamco.cd.training.common.utils.UserUtil;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import lombok.RequiredArgsConstructor;
import org.springframework.http.HttpHeaders;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Service;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
@Service
@RequiredArgsConstructor
public class DownloadExecutor {
private final UserUtil userUtil;
public ResponseEntity<StreamingResponseBody> stream(DownloadSpec spec) throws IOException {
if (!Files.isReadable(spec.filePath())) {
return ResponseEntity.notFound().build();
}
StreamingResponseBody body =
os -> {
try (InputStream in = Files.newInputStream(spec.filePath())) {
in.transferTo(os);
os.flush();
} catch (Exception e) {
// 고용량은 중간 끊김 흔하니까 throw 금지
}
};
String fileName =
spec.downloadName() != null
? spec.downloadName()
: spec.filePath().getFileName().toString();
return ResponseEntity.ok()
.contentType(
spec.contentType() != null ? spec.contentType() : MediaType.APPLICATION_OCTET_STREAM)
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + fileName + "\"")
.body(body);
}
}

View File

@@ -0,0 +1,19 @@
package com.kamco.cd.training.common.download;
import org.springframework.util.AntPathMatcher;
public final class DownloadPaths {
private DownloadPaths() {}
public static final String[] PATTERNS = {
"/api/inference/download/**", "/api/training-data/stage/download/**"
};
public static boolean matches(String uri) {
AntPathMatcher m = new AntPathMatcher();
for (String p : PATTERNS) {
if (m.match(p, uri)) return true;
}
return false;
}
}

View File

@@ -0,0 +1,81 @@
package com.kamco.cd.training.common.download;
import jakarta.servlet.http.HttpServletRequest;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;
import org.springframework.core.io.FileSystemResource;
import org.springframework.core.io.Resource;
import org.springframework.core.io.support.ResourceRegion;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpRange;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Component;
@Component
public class RangeDownloadResponder {
public ResponseEntity<?> buildZipResponse(
Path filePath, String downloadFileName, HttpServletRequest request) throws IOException {
if (!Files.isRegularFile(filePath)) {
return ResponseEntity.notFound().build();
}
long totalSize = Files.size(filePath);
Resource resource = new FileSystemResource(filePath);
String disposition = "attachment; filename=\"" + downloadFileName + "\"";
String rangeHeader = request.getHeader(HttpHeaders.RANGE);
// 🔥 공통 헤더 (여기 고정)
ResponseEntity.BodyBuilder base =
ResponseEntity.ok()
.contentType(MediaType.APPLICATION_OCTET_STREAM)
.header(HttpHeaders.CONTENT_DISPOSITION, disposition)
.header(HttpHeaders.ACCEPT_RANGES, "bytes")
.header("Access-Control-Expose-Headers", "Content-Disposition")
.header("X-Accel-Buffering", "no");
if (rangeHeader == null || rangeHeader.isBlank()) {
return base.contentLength(totalSize).body(resource);
}
List<HttpRange> ranges;
try {
ranges = HttpRange.parseRanges(rangeHeader);
} catch (IllegalArgumentException ex) {
return ResponseEntity.status(416)
.header(HttpHeaders.CONTENT_RANGE, "bytes */" + totalSize)
.header("X-Accel-Buffering", "no")
.build();
}
HttpRange range = ranges.get(0);
long start = range.getRangeStart(totalSize);
long end = range.getRangeEnd(totalSize);
if (start >= totalSize) {
return ResponseEntity.status(416)
.header(HttpHeaders.CONTENT_RANGE, "bytes */" + totalSize)
.header("X-Accel-Buffering", "no")
.build();
}
long regionLength = end - start + 1;
ResourceRegion region = new ResourceRegion(resource, start, regionLength);
return ResponseEntity.status(206)
.contentType(MediaType.APPLICATION_OCTET_STREAM)
.header(HttpHeaders.CONTENT_DISPOSITION, disposition)
.header(HttpHeaders.ACCEPT_RANGES, "bytes")
.header("Access-Control-Expose-Headers", "Content-Disposition")
.header("X-Accel-Buffering", "no")
.header(HttpHeaders.CONTENT_RANGE, "bytes " + start + "-" + end + "/" + totalSize)
.contentLength(regionLength)
.body(region);
}
}

View File

@@ -0,0 +1,12 @@
package com.kamco.cd.training.common.download.dto;
import java.nio.file.Path;
import java.util.UUID;
import org.springframework.http.MediaType;
public record DownloadSpec(
UUID uuid, // 다운로드 식별(로그/정책용)
Path filePath, // 실제 파일 경로
String downloadName, // 사용자에게 보일 파일명
MediaType contentType // 보통 OCTET_STREAM
) {}

View File

@@ -1,5 +1,6 @@
package com.kamco.cd.training.common.dto; package com.kamco.cd.training.common.dto;
import com.kamco.cd.training.common.enums.ModelType;
import io.swagger.v3.oas.annotations.media.Schema; import io.swagger.v3.oas.annotations.media.Schema;
import lombok.AllArgsConstructor; import lombok.AllArgsConstructor;
import lombok.Getter; import lombok.Getter;
@@ -11,9 +12,14 @@ import lombok.Setter;
@AllArgsConstructor @AllArgsConstructor
@NoArgsConstructor @NoArgsConstructor
public class HyperParam { public class HyperParam {
@Schema(description = "모델", example = "G1")
private ModelType model; // G1, G2, G3
// ------------------------- // -------------------------
// Important // Important
// ------------------------- // -------------------------
@Schema(description = "백본 네트워크", example = "large") @Schema(description = "백본 네트워크", example = "large")
private String backbone; // backbone private String backbone; // backbone

View File

@@ -0,0 +1,8 @@
package com.kamco.cd.training.common.dto;
public class MonitorDto {
public int cpu; // CPU 사용률 (%)
public long[] memory; // "사용/전체"
public int gpu; // 🔥 전체 GPU 평균 (%)
}

View File

@@ -0,0 +1,27 @@
package com.kamco.cd.training.common.enums;
import com.kamco.cd.training.common.utils.enums.EnumType;
import lombok.AllArgsConstructor;
import lombok.Getter;
@Getter
@AllArgsConstructor
public enum JobStatusType implements EnumType {
QUEUED("대기중"),
RUNNING("실행중"),
SUCCESS("성공"),
FAILED("실패"),
CANCELED("취소");
private final String desc;
@Override
public String getId() {
return name();
}
@Override
public String getText() {
return desc;
}
}

View File

@@ -0,0 +1,24 @@
package com.kamco.cd.training.common.enums;
import com.kamco.cd.training.common.utils.enums.EnumType;
import lombok.AllArgsConstructor;
import lombok.Getter;
@Getter
@AllArgsConstructor
public enum JobType implements EnumType {
TRAIN("학습"),
TEST("테스트");
private final String desc;
@Override
public String getId() {
return name();
}
@Override
public String getText() {
return desc;
}
}

View File

@@ -2,6 +2,7 @@ package com.kamco.cd.training.common.enums;
import com.kamco.cd.training.common.utils.enums.CodeExpose; import com.kamco.cd.training.common.utils.enums.CodeExpose;
import com.kamco.cd.training.common.utils.enums.EnumType; import com.kamco.cd.training.common.utils.enums.EnumType;
import java.util.Arrays;
import lombok.AllArgsConstructor; import lombok.AllArgsConstructor;
import lombok.Getter; import lombok.Getter;
@@ -15,6 +16,13 @@ public enum ModelType implements EnumType {
private String desc; private String desc;
public static ModelType getValueData(String modelNo) {
return Arrays.stream(ModelType.values())
.filter(m -> m.getId().equals(modelNo))
.findFirst()
.orElse(G1);
}
@Override @Override
public String getId() { public String getId() {
return name(); return name();

View File

@@ -0,0 +1,142 @@
package com.kamco.cd.training.common.service;
import jakarta.annotation.PostConstruct;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import lombok.extern.log4j.Log4j2;
import org.springframework.stereotype.Component;
@Component
@Log4j2
public class GpuDmonReader {
// =========================
// GPU 사용률 저장소
// key: GPU index (0,1,2...)
// value: 현재 GPU 사용률 (%)
// ConcurrentHashMap → 멀티스레드 안전
// =========================
private final Map<Integer, Integer> gpuUtilMap = new ConcurrentHashMap<>();
// =========================
// 외부 조회용
// SystemMonitorService에서 호출
// =========================
public Map<Integer, Integer> getGpuUtilMap() {
return gpuUtilMap;
}
// =========================
// Bean 초기화 시 실행
// - 별도 스레드에서 GPU 모니터링 시작
// - 메인 스레드 block 방지
// =========================
@PostConstruct
public void start() {
// nvidia-smi 없는 환경이면 GPU 모니터링 비활성화
if (!isNvidiaAvailable()) {
log.warn("nvidia-smi not found. GPU monitoring disabled.");
return;
}
// 데몬 스레드로 실행 (서버 종료 시 자동 종료)
Thread t = new Thread(this::runLoop, "gpu-dmon-thread");
t.setDaemon(true);
t.start();
}
// =========================
// 무한 루프
// - dmon 실행
// - 죽으면 자동 재시작
// =========================
private void runLoop() {
while (true) {
try {
runDmon(); // GPU 사용률 수집 시작
} catch (Exception e) {
// dmon 프로세스 종료되면 여기로 들어옴
log.warn("dmon restart: {}", e.getMessage());
}
// 5초 대기 후 재시작
sleep(5000);
}
}
// =========================
// nvidia-smi dmon 실행
// - GPU 사용률 스트리밍으로 계속 수신
// =========================
private void runDmon() throws Exception {
// -s u → GPU utilization만 출력
ProcessBuilder pb = new ProcessBuilder("nvidia-smi", "dmon", "-s", "u");
// 프로세스 실행 후 stdout 읽기
try (BufferedReader br =
new BufferedReader(new InputStreamReader(pb.start().getInputStream()))) {
String line;
// dmon은 계속 출력됨 (스트리밍)
while ((line = br.readLine()) != null) {
// 헤더 제거 (#로 시작)
if (line.startsWith("#")) continue;
line = line.trim();
if (line.isEmpty()) continue;
// 공백 기준 분리
String[] parts = line.split("\\s+");
// 첫 번째 값이 GPU index인지 확인
if (!parts[0].matches("\\d+")) continue;
int index = Integer.parseInt(parts[0]);
try {
// 두 번째 값이 GPU 사용률 (sm)
int util = Integer.parseInt(parts[1]);
// 최신 값 갱신
gpuUtilMap.put(index, util);
} catch (Exception ignored) {
// 파싱 실패 시 무시
}
}
}
// 여기까지 왔다는 건 dmon 프로세스 종료됨
// → runLoop에서 재시작하도록 예외 발생
throw new IllegalStateException("dmon stopped");
}
// =========================
// nvidia-smi 존재 여부 확인
// =========================
private boolean isNvidiaAvailable() {
try {
Process p = new ProcessBuilder("which", "nvidia-smi").start();
return p.waitFor() == 0;
} catch (Exception e) {
return false;
}
}
// =========================
// sleep 유틸
// =========================
private void sleep(long ms) {
try {
Thread.sleep(ms);
} catch (InterruptedException ignored) {
}
}
}

View File

@@ -0,0 +1,224 @@
package com.kamco.cd.training.common.service;
import com.kamco.cd.training.common.dto.MonitorDto;
import java.io.BufferedReader;
import java.io.FileReader;
import java.util.ArrayDeque;
import java.util.Deque;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import lombok.RequiredArgsConstructor;
import lombok.extern.log4j.Log4j2;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Service;
@Service
@RequiredArgsConstructor
@Log4j2
public class SystemMonitorService {
// =========================
// CPU 이전값 (delta 계산용)
// - /proc/stat은 누적값이기 때문에
// - 이전 값과 비교해서 사용률 계산
// =========================
private long prevIdle = 0;
private long prevTotal = 0;
// =========================
// 최근 30초 히스토리
// - CPU: 30개 (1초 * 30)
// - GPU: GPU별 30개
// =========================
private final Deque<Double> cpuHistory = new ArrayDeque<>();
// key: GPU index
// value: 최근 30개 사용률
private final Map<Integer, Deque<Integer>> gpuHistory = new ConcurrentHashMap<>();
// =========================
// GPU 데이터 제공 (dmon reader)
// =========================
private final GpuDmonReader gpuReader;
// =========================
// 캐시 (API 응답용)
// - 매 요청마다 계산하지 않기 위해 사용
// - volatile → 멀티스레드 안전하게 최신값 유지
// =========================
private volatile MonitorDto cached = new MonitorDto();
// =========================
// 1초마다 수집
// =========================
@Scheduled(fixedRate = 1000)
public void collect() {
try {
// =====================
// 1. CPU 수집
// =====================
double cpu = readCpu();
cpuHistory.add(cpu);
// 30개 유지 (rolling window)
if (cpuHistory.size() > 30) cpuHistory.poll();
// =====================
// 2. GPU 수집
// =====================
Map<Integer, Integer> gpuMap = gpuReader.getGpuUtilMap();
for (Map.Entry<Integer, Integer> entry : gpuMap.entrySet()) {
int index = entry.getKey();
int util = entry.getValue();
// GPU별 히스토리 생성 및 추가
gpuHistory.computeIfAbsent(index, k -> new ArrayDeque<>()).add(util);
// 30개 유지
Deque<Integer> q = gpuHistory.get(index);
if (q.size() > 30) q.poll();
}
// =====================
// 3. 캐시 업데이트
// =====================
updateCache();
} catch (Exception e) {
log.error("collect error", e);
}
}
// =========================
// CPU 사용률 계산
// - /proc/stat 사용
// - 이전값과의 차이로 계산 (delta 방식)
// =========================
private double readCpu() throws Exception {
if (!isLinux()) return 0;
try (BufferedReader br = new BufferedReader(new FileReader("/proc/stat"))) {
String[] p = br.readLine().split("\\s+");
long user = Long.parseLong(p[1]);
long nice = Long.parseLong(p[2]);
long system = Long.parseLong(p[3]);
long idle = Long.parseLong(p[4]);
long iowait = Long.parseLong(p[5]);
long irq = Long.parseLong(p[6]);
long softirq = Long.parseLong(p[7]);
long total = user + nice + system + idle + iowait + irq + softirq;
long idleAll = idle + iowait;
// 최초 실행 시 기준값만 저장
if (prevTotal == 0) {
prevTotal = total;
prevIdle = idleAll;
return 0;
}
long totalDiff = total - prevTotal;
long idleDiff = idleAll - prevIdle;
prevTotal = total;
prevIdle = idleAll;
if (totalDiff == 0) return 0;
// CPU 사용률 (%)
return (1.0 - (double) idleDiff / totalDiff) * 100;
}
}
// =========================
// Linux 환경 체크
// =========================
private boolean isLinux() {
return System.getProperty("os.name").toLowerCase().contains("linux");
}
// =========================
// Memory 조회 (/proc/meminfo)
// - OS 값 그대로 사용 (kB)
// - [사용량, 전체]
// =========================
private long[] readMemory() throws Exception {
if (!isLinux()) return new long[] {0, 0};
try (BufferedReader br = new BufferedReader(new FileReader("/proc/meminfo"))) {
long total = 0;
long available = 0;
String line;
while ((line = br.readLine()) != null) {
if (line.startsWith("MemTotal")) {
total = Long.parseLong(line.replaceAll("\\D+", ""));
} else if (line.startsWith("MemAvailable")) {
available = Long.parseLong(line.replaceAll("\\D+", ""));
}
}
long used = total - available;
return new long[] {used, total};
}
}
// =========================
// 캐시 업데이트
// - CPU: 30초 평균
// - GPU: 전체 샘플 평균
// - Memory: 현재값
// =========================
private void updateCache() throws Exception {
MonitorDto dto = new MonitorDto();
// =====================
// CPU 평균 (30초)
// =====================
dto.cpu = (int) cpuHistory.stream().mapToDouble(Double::doubleValue).average().orElse(0);
// =====================
// Memory (kB 그대로)
// =====================
dto.memory = readMemory();
// =====================
// GPU 평균 (🔥 전체 샘플 기준)
// =====================
int sum = 0;
int count = 0;
for (Deque<Integer> q : gpuHistory.values()) {
for (int v : q) {
sum += v;
count++;
}
}
dto.gpu = (count == 0) ? 0 : sum / count;
// =====================
// 캐시 교체 (atomic)
// =====================
this.cached = dto;
}
// =========================
// 외부 조회
// - Controller에서 호출
// =========================
public MonitorDto get() {
return cached;
}
}

View File

@@ -2,6 +2,11 @@ package com.kamco.cd.training.common.utils;
import static java.lang.String.CASE_INSENSITIVE_ORDER; import static java.lang.String.CASE_INSENSITIVE_ORDER;
import com.jcraft.jsch.ChannelExec;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.Session;
import com.kamco.cd.training.common.exception.CustomApiException;
import com.kamco.cd.training.config.api.ApiResponseDto.ApiResponseCode;
import io.swagger.v3.oas.annotations.media.Schema; import io.swagger.v3.oas.annotations.media.Schema;
import java.io.BufferedReader; import java.io.BufferedReader;
import java.io.File; import java.io.File;
@@ -11,6 +16,7 @@ import java.io.FileReader;
import java.io.IOException; import java.io.IOException;
import java.io.InputStream; import java.io.InputStream;
import java.io.InputStreamReader; import java.io.InputStreamReader;
import java.io.UncheckedIOException;
import java.nio.file.Files; import java.nio.file.Files;
import java.nio.file.Path; import java.nio.file.Path;
import java.nio.file.Paths; import java.nio.file.Paths;
@@ -23,6 +29,7 @@ import java.util.Arrays;
import java.util.Comparator; import java.util.Comparator;
import java.util.Date; import java.util.Date;
import java.util.List; import java.util.List;
import java.util.Properties;
import java.util.Set; import java.util.Set;
import java.util.function.Predicate; import java.util.function.Predicate;
import java.util.stream.Collectors; import java.util.stream.Collectors;
@@ -34,6 +41,7 @@ import lombok.extern.slf4j.Slf4j;
import org.apache.commons.io.FilenameUtils; import org.apache.commons.io.FilenameUtils;
import org.geotools.coverage.grid.GridCoverage2D; import org.geotools.coverage.grid.GridCoverage2D;
import org.geotools.gce.geotiff.GeoTiffReader; import org.geotools.gce.geotiff.GeoTiffReader;
import org.springframework.http.HttpStatus;
import org.springframework.util.FileSystemUtils; import org.springframework.util.FileSystemUtils;
import org.springframework.web.multipart.MultipartFile; import org.springframework.web.multipart.MultipartFile;
@@ -501,11 +509,15 @@ public class FIleChecker {
try { try {
File dir = new File(targetPath); File dir = new File(targetPath);
log.info("targetPath={}", targetPath);
log.info("absolute targetPath={}", dir.getAbsolutePath());
if (!dir.exists()) { if (!dir.exists()) {
dir.mkdirs(); dir.mkdirs();
} }
File dest = new File(dir, String.valueOf(chunkIndex)); File dest = new File(dir, String.valueOf(chunkIndex));
log.info("real save path = {}", dest.getAbsolutePath());
log.info("chunkIndex={}, uploadSize={}", chunkIndex, mfile.getSize()); log.info("chunkIndex={}, uploadSize={}", chunkIndex, mfile.getSize());
log.info("savedSize={}", dest.length()); log.info("savedSize={}", dest.length());
@@ -517,6 +529,9 @@ public class FIleChecker {
log.info("after delete={}", dest.length()); log.info("after delete={}", dest.length());
mfile.transferTo(dest); mfile.transferTo(dest);
log.info("after transfer size={}", dest.length());
log.info("after transfer exists={}", dest.exists());
return true; return true;
} catch (IOException e) { } catch (IOException e) {
log.error("chunk save error", e); log.error("chunk save error", e);
@@ -702,12 +717,30 @@ public class FIleChecker {
} }
public static void unzip(String fileName, String destDirectory) throws IOException { public static void unzip(String fileName, String destDirectory) throws IOException {
File destDir = new File(destDirectory); String zipFilePath = destDirectory + File.separator + fileName;
if (!destDir.exists()) {
destDir.mkdirs(); // 대상 폴더가 없으면 생성 log.info("fileName : {}", fileName);
log.info("destDirectory : {}", destDirectory);
log.info("zipFilePath : {}", zipFilePath);
// zip 이름으로 폴더 생성 (확장자 제거)
String folderName =
fileName.endsWith(".zip") ? fileName.substring(0, fileName.length() - 4) : fileName;
log.info("folderName : {}", folderName);
File destDir = new File(destDirectory, folderName);
log.info("destDir : {}", destDir);
// 동일 폴더가 이미 있으면 삭제
log.info("111 destDir.exists() : {}", destDir.exists());
if (destDir.exists()) {
deleteDirectoryRecursively(destDir.toPath());
} }
String zipFilePath = destDirectory + "/" + fileName; log.info("222 destDir.exists() : {}", destDir.exists());
if (!destDir.exists()) {
log.info("mkdirs : {}", destDir.exists());
destDir.mkdirs();
}
try (ZipInputStream zis = new ZipInputStream(new FileInputStream(zipFilePath))) { try (ZipInputStream zis = new ZipInputStream(new FileInputStream(zipFilePath))) {
ZipEntry zipEntry = zis.getNextEntry(); ZipEntry zipEntry = zis.getNextEntry();
@@ -740,6 +773,11 @@ public class FIleChecker {
zipEntry = zis.getNextEntry(); zipEntry = zis.getNextEntry();
} }
zis.closeEntry(); zis.closeEntry();
} catch (IOException e) {
throw new CustomApiException(
ApiResponseCode.INTERNAL_SERVER_ERROR.getId(),
HttpStatus.INTERNAL_SERVER_ERROR,
"압축 해제 중 오류가 발생했습니다: " + e.getMessage());
} }
} }
@@ -755,4 +793,70 @@ public class FIleChecker {
return destFile; return destFile;
} }
public static List<String> execCommandAndReadLines(String command) {
List<String> result = new ArrayList<>();
String host = "192.168.2.86";
String user = "kcomu";
String password = "Kamco2025!";
Session session = null;
ChannelExec channel = null;
try {
JSch jsch = new JSch();
session = jsch.getSession(user, host, 22);
session.setPassword(password);
Properties config = new Properties();
config.put("StrictHostKeyChecking", "no");
session.setConfig(config);
session.connect(10_000);
channel = (ChannelExec) session.openChannel("exec");
channel.setCommand(command);
channel.setInputStream(null);
InputStream in = channel.getInputStream();
channel.connect();
try (BufferedReader br = new BufferedReader(new InputStreamReader(in))) {
String line;
while ((line = br.readLine()) != null) {
result.add(line);
}
}
return result;
} catch (Exception e) {
throw new RuntimeException("remote command failed : " + command, e);
} finally {
if (channel != null) channel.disconnect();
if (session != null) session.disconnect();
}
}
/** ✅ 폴더 재귀 삭제 */
private static void deleteDirectoryRecursively(Path path) throws IOException {
if (!Files.exists(path)) return;
// 하위부터 지워야 하므로 reverse order
Files.walk(path)
.sorted(Comparator.reverseOrder())
.forEach(
p -> {
try {
Files.deleteIfExists(p);
} catch (IOException e) {
// 여기서 바로 RuntimeException으로 올려서 상위 catch(IOException)로 잡히게 함
throw new UncheckedIOException("폴더 삭제 실패: " + p.toAbsolutePath(), e);
}
});
}
} }

View File

@@ -0,0 +1,23 @@
package com.kamco.cd.training.common.utils;
import jakarta.servlet.http.HttpServletRequest;
public final class HeaderUtil {
private HeaderUtil() {}
/** 특정 Header 값 조회 */
public static String get(HttpServletRequest request, String headerName) {
if (request == null || headerName == null) {
return null;
}
String value = request.getHeader(headerName);
return (value != null && !value.isBlank()) ? value : null;
}
/** 필수 Header 조회 (없으면 null) */
public static String getRequired(HttpServletRequest request, String headerName) {
return get(request, headerName);
}
}

View File

@@ -0,0 +1,23 @@
package com.kamco.cd.training.config;
import java.util.concurrent.Executor;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
@Configuration
@EnableAsync
public class AsyncConfig {
@Bean(name = "trainJobExecutor")
public Executor trainJobExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(4); // 동시에 4개 실행
executor.setMaxPoolSize(8); // 최대 8개
executor.setQueueCapacity(200); // 대기 큐
executor.setThreadNamePrefix("train-job-");
executor.initialize();
return executor;
}
}

View File

@@ -76,11 +76,13 @@ public class SecurityConfig {
"/api/auth/logout", "/api/auth/logout",
"/swagger-ui/**", "/swagger-ui/**",
"/v3/api-docs/**", "/v3/api-docs/**",
"/api/members/*/password",
"/api/upload/chunk-upload-dataset", "/api/upload/chunk-upload-dataset",
"/api/upload/chunk-upload-complete") "/api/upload/chunk-upload-complete",
"/download_progress_test.html",
"/api/models/download/**")
.permitAll() .permitAll()
.requestMatchers("/api/members/*/password")
.authenticated()
// default // default
.anyRequest() .anyRequest()
.authenticated()) .authenticated())
@@ -102,15 +104,19 @@ public class SecurityConfig {
return new BCryptPasswordEncoder(); return new BCryptPasswordEncoder();
} }
/** CORS 설정 */ /** CORS 설정 - application.yml에서 환경별로 관리 */
@Bean @Bean
public CorsConfigurationSource corsConfigurationSource() { public CorsConfigurationSource corsConfigurationSource() {
CorsConfiguration config = new CorsConfiguration(); // CORS 객체 생성 CorsConfiguration config = new CorsConfiguration(); // CORS 객체 생성
// application.yml에서 환경별로 설정된 도메인 사용
config.setAllowedOriginPatterns(List.of("*")); // 도메인 허용 config.setAllowedOriginPatterns(List.of("*")); // 도메인 허용
config.setAllowedMethods(List.of("GET", "POST", "PUT", "PATCH", "DELETE", "OPTIONS")); config.setAllowedMethods(List.of("GET", "POST", "PUT", "PATCH", "DELETE", "OPTIONS"));
config.setAllowedHeaders(List.of("*")); // 헤더요청 Authorization, Content-Type, X-Custom-Header config.setAllowedHeaders(List.of("*")); // 헤더요청 Authorization, Content-Type, X-Custom-Header
config.setAllowCredentials(true); // 쿠키, Authorization 헤더, Bearer Token 등 자격증명 포함 요청을 허용할지 설정 config.setAllowCredentials(true); // 쿠키, Authorization 헤더, Bearer Token 등 자격증명 포함 요청을 허용할지 설정
config.setExposedHeaders(List.of("Content-Disposition")); config.setExposedHeaders(List.of("Content-Disposition", "Authorization"));
config.setMaxAge(3600L); // Preflight 요청 캐시 (1시간)
UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource(); UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource();
/** "/**" → 모든 API 경로에 대해 이 CORS 규칙을 적용 /api/** 같이 특정 경로만 지정 가능. */ /** "/**" → 모든 API 경로에 대해 이 CORS 규칙을 적용 /api/** 같이 특정 경로만 지정 가능. */

View File

@@ -57,7 +57,7 @@ public class StartupLogger {
""" """
╔════════════════════════════════════════════════════════════════════════════════╗ ╔════════════════════════════════════════════════════════════════════════════════╗
║ 🚀 APPLICATION STARTUP INFORMATION ║ 🚀 APPLICATION STARTUP INFORMATION 2
╠════════════════════════════════════════════════════════════════════════════════╣ ╠════════════════════════════════════════════════════════════════════════════════╣
║ PROFILE CONFIGURATION ║ ║ PROFILE CONFIGURATION ║
╠────────────────────────────────────────────────────────────────────────────────╣ ╠────────────────────────────────────────────────────────────────────────────────╣

View File

@@ -5,11 +5,14 @@ import com.kamco.cd.training.log.dto.EventType;
import com.kamco.cd.training.menu.dto.MenuDto; import com.kamco.cd.training.menu.dto.MenuDto;
import jakarta.servlet.http.HttpServletRequest; import jakarta.servlet.http.HttpServletRequest;
import java.io.UnsupportedEncodingException; import java.io.UnsupportedEncodingException;
import java.util.Comparator;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
import java.util.stream.Collectors; import java.util.stream.Collectors;
import lombok.extern.slf4j.Slf4j;
import org.springframework.web.util.ContentCachingRequestWrapper; import org.springframework.web.util.ContentCachingRequestWrapper;
@Slf4j
public class ApiLogFunction { public class ApiLogFunction {
// 클라이언트 IP 추출 // 클라이언트 IP 추출
@@ -34,6 +37,14 @@ public class ApiLogFunction {
return ip; return ip;
} }
public static String getXFowardedForIp(HttpServletRequest request) {
String ip = request.getHeader("X-Forwarded-For");
if (ip != null) {
ip = ip.split(",")[0].trim();
}
return ip;
}
// 사용자 ID 추출 예시 (Spring Security 기준) // 사용자 ID 추출 예시 (Spring Security 기준)
public static String getUserId(HttpServletRequest request) { public static String getUserId(HttpServletRequest request) {
try { try {
@@ -47,20 +58,20 @@ public class ApiLogFunction {
String method = request.getMethod().toUpperCase(); String method = request.getMethod().toUpperCase();
String uri = request.getRequestURI().toLowerCase(); String uri = request.getRequestURI().toLowerCase();
// URL 기반 DOWNLOAD/PRINT 분류 // URL 기반 DOWNLOAD/PRINT 분류 -> /download는 FileDownloadInterceptor로 옮김
if (uri.contains("/download") || uri.contains("/export")) { if (uri.contains("/download") || uri.contains("/export")) {
return EventType.DOWNLOAD; return EventType.DOWNLOAD;
} }
if (uri.contains("/print")) { if (uri.contains("/print")) {
return EventType.PRINT; return EventType.OTHER;
} }
// 일반 CRUD // 일반 CRUD
return switch (method) { return switch (method) {
case "POST" -> EventType.CREATE; case "POST" -> EventType.ADDED;
case "GET" -> EventType.READ; case "GET" -> EventType.LIST;
case "DELETE" -> EventType.DELETE; case "DELETE" -> EventType.REMOVE;
case "PUT", "PATCH" -> EventType.UPDATE; case "PUT", "PATCH" -> EventType.MODIFIED;
default -> EventType.OTHER; default -> EventType.OTHER;
}; };
} }
@@ -121,12 +132,22 @@ public class ApiLogFunction {
public static String getUriMenuInfo(List<MenuDto.Basic> menuList, String uri) { public static String getUriMenuInfo(List<MenuDto.Basic> menuList, String uri) {
MenuDto.Basic m = String normalizedUri = uri.replace("/api", "");
MenuDto.Basic basic =
menuList.stream() menuList.stream()
.filter(menu -> menu.getMenuApiUrl() != null && uri.contains(menu.getMenuApiUrl())) .filter(
.findFirst() menu -> menu.getMenuUrl() != null && normalizedUri.startsWith(menu.getMenuUrl()))
.max(Comparator.comparingInt(m -> m.getMenuUrl().length()))
.orElse(null); .orElse(null);
return m != null ? m.getMenuUid() : "SYSTEM"; return basic != null ? basic.getMenuUid() : "SYSTEM";
}
public static String cutRequestBody(String value) {
int MAX_LEN = 255;
if (value == null) {
return null;
}
return value.length() <= MAX_LEN ? value : value.substring(0, MAX_LEN);
} }
} }

View File

@@ -2,10 +2,17 @@ package com.kamco.cd.training.config.api;
import com.fasterxml.jackson.databind.ObjectMapper; import com.fasterxml.jackson.databind.ObjectMapper;
import com.kamco.cd.training.auth.CustomUserDetails; import com.kamco.cd.training.auth.CustomUserDetails;
import com.kamco.cd.training.common.utils.HeaderUtil;
import com.kamco.cd.training.log.dto.EventType;
import com.kamco.cd.training.menu.dto.MenuDto;
import com.kamco.cd.training.menu.service.MenuService; import com.kamco.cd.training.menu.service.MenuService;
import com.kamco.cd.training.postgres.entity.AuditLogEntity; import com.kamco.cd.training.postgres.entity.AuditLogEntity;
import com.kamco.cd.training.postgres.repository.log.AuditLogRepository; import com.kamco.cd.training.postgres.repository.log.AuditLogRepository;
import jakarta.servlet.http.HttpServletRequest; import jakarta.servlet.http.HttpServletRequest;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Optional;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.MethodParameter; import org.springframework.core.MethodParameter;
import org.springframework.http.MediaType; import org.springframework.http.MediaType;
@@ -23,6 +30,7 @@ import org.springframework.web.util.ContentCachingRequestWrapper;
* *
* <p>createOK() → 201 CREATED ok() → 200 OK deleteOk() → 204 NO_CONTENT * <p>createOK() → 201 CREATED ok() → 200 OK deleteOk() → 204 NO_CONTENT
*/ */
@Slf4j
@RestControllerAdvice @RestControllerAdvice
public class ApiResponseAdvice implements ResponseBodyAdvice<Object> { public class ApiResponseAdvice implements ResponseBodyAdvice<Object> {
@@ -61,12 +69,27 @@ public class ApiResponseAdvice implements ResponseBodyAdvice<Object> {
if (body instanceof ApiResponseDto<?> apiResponse) { if (body instanceof ApiResponseDto<?> apiResponse) {
response.setStatusCode(apiResponse.getHttpStatus()); response.setStatusCode(apiResponse.getHttpStatus());
String ip = ApiLogFunction.getClientIp(servletRequest); String actionType = HeaderUtil.get(servletRequest, "kamco-action-type");
Long userid = null; // actionType 이 없으면 로그 저장하지 않기 || download 는 FileDownloadInterceptor 에서 하기
// (file down URL prefix 추가는 WebConfig.java 에 하기)
if (actionType == null || actionType.equalsIgnoreCase("download")) {
return body;
}
if (servletRequest.getUserPrincipal() instanceof UsernamePasswordAuthenticationToken auth String ip =
&& auth.getPrincipal() instanceof CustomUserDetails customUserDetails) { Optional.ofNullable(HeaderUtil.get(servletRequest, "kamco-user-ip"))
userid = customUserDetails.getMember().getId(); .orElseGet(() -> ApiLogFunction.getXFowardedForIp(servletRequest));
Long userid = null;
String loginAttemptId = null;
// 로그인 시도할 때
if (servletRequest.getRequestURI().contains("/api/auth/signin")) {
loginAttemptId = HeaderUtil.get(servletRequest, "kamco-login-attempt-id");
} else {
if (servletRequest.getUserPrincipal() instanceof UsernamePasswordAuthenticationToken auth
&& auth.getPrincipal() instanceof CustomUserDetails customUserDetails) {
userid = customUserDetails.getMember().getId();
}
} }
String requestBody; String requestBody;
@@ -84,17 +107,33 @@ public class ApiResponseAdvice implements ResponseBodyAdvice<Object> {
requestBody = maskSensitiveFields(requestBody); requestBody = maskSensitiveFields(requestBody);
} }
List<?> list = menuService.getFindAll();
List<MenuDto.Basic> result =
list.stream()
.map(
item -> {
if (item instanceof LinkedHashMap<?, ?> map) {
return objectMapper.convertValue(map, MenuDto.Basic.class);
} else if (item instanceof MenuDto.Basic dto) {
return dto;
} else {
throw new IllegalStateException("Unsupported cache type: " + item.getClass());
}
})
.toList();
AuditLogEntity log = AuditLogEntity log =
new AuditLogEntity( new AuditLogEntity(
userid, userid,
ApiLogFunction.getEventType(servletRequest), EventType.fromName(actionType),
ApiLogFunction.isSuccessFail(apiResponse), ApiLogFunction.isSuccessFail(apiResponse),
ApiLogFunction.getUriMenuInfo( ApiLogFunction.getUriMenuInfo(result, servletRequest.getRequestURI()),
menuService.getFindAll(), servletRequest.getRequestURI()),
ip, ip,
servletRequest.getRequestURI(), servletRequest.getRequestURI(),
requestBody, ApiLogFunction.cutRequestBody(requestBody),
apiResponse.getErrorLogUid()); apiResponse.getErrorLogUid(),
null,
loginAttemptId);
auditLogRepository.save(log); auditLogRepository.save(log);
} }

View File

@@ -14,15 +14,15 @@ import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.responses.ApiResponses; import io.swagger.v3.oas.annotations.responses.ApiResponses;
import io.swagger.v3.oas.annotations.tags.Tag; import io.swagger.v3.oas.annotations.tags.Tag;
import jakarta.validation.Valid; import jakarta.validation.Valid;
import java.io.IOException;
import java.nio.file.FileStore;
import java.nio.file.Files;
import java.nio.file.Path; import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.List; import java.util.List;
import java.util.UUID; import java.util.UUID;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.springframework.core.io.Resource; import org.springframework.core.io.Resource;
import org.springframework.core.io.UrlResource;
import org.springframework.data.domain.Page; import org.springframework.data.domain.Page;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity; import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*; import org.springframework.web.bind.annotation.*;
@@ -212,8 +212,15 @@ public class DatasetApiController {
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content) @ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
}) })
@GetMapping("/usable-bytes") @GetMapping("/usable-bytes")
public ApiResponseDto<DatasetStorage> getUsableBytes() { public ApiResponseDto<DatasetStorage> getUsableBytes() throws IOException {
return ApiResponseDto.ok(datasetService.getUsableBytes()); FileStore store = Files.getFileStore(Path.of("."));
long usable = store.getUsableSpace();
DatasetStorage storage = new DatasetStorage();
storage.setUsableBytes(String.valueOf(usable));
// datasetService.getUsableBytes();
return ApiResponseDto.ok(storage);
} }
@Operation(summary = "학습데이터 zip파일 등록", description = "학습데이터 zip파일 등록 합니다.") @Operation(summary = "학습데이터 zip파일 등록", description = "학습데이터 zip파일 등록 합니다.")
@@ -221,7 +228,7 @@ public class DatasetApiController {
public ApiResponseDto<ApiResponseDto.ResponseObj> insertDataset( public ApiResponseDto<ApiResponseDto.ResponseObj> insertDataset(
@RequestBody @Valid DatasetDto.AddReq addReq) { @RequestBody @Valid DatasetDto.AddReq addReq) {
return ApiResponseDto.ok(datasetService.insertDataset(addReq)); return ApiResponseDto.okObject(datasetService.insertDataset(addReq));
} }
@Operation(summary = "객체별 파일 Path 조회", description = "파일 Path 조회") @Operation(summary = "객체별 파일 Path 조회", description = "파일 Path 조회")
@@ -230,10 +237,15 @@ public class DatasetApiController {
throws Exception { throws Exception {
String path = datasetService.getFilePathByUUIDPathType(uuid, pathType); String path = datasetService.getFilePathByUUIDPathType(uuid, pathType);
Path filePath = Paths.get(path); return datasetService.getFilePathByFile(path);
}
Resource resource = new UrlResource(filePath.toUri()); @Operation(summary = "객체별 파일 Path 조회", description = "파일 Path 조회")
@GetMapping("/files-to86")
public ResponseEntity<Resource> getFileTo86(
@RequestParam UUID uuid, @RequestParam String pathType) throws Exception {
return ResponseEntity.ok().contentType(MediaType.APPLICATION_OCTET_STREAM).body(resource); String path = datasetService.getFilePathByUUIDPathType(uuid, pathType);
return datasetService.getFilePathByFile(path);
} }
} }

View File

@@ -1,8 +1,9 @@
package com.kamco.cd.training.dataset.dto; package com.kamco.cd.training.dataset.dto;
import com.fasterxml.jackson.annotation.JsonInclude; import com.fasterxml.jackson.annotation.JsonIgnore;
import com.kamco.cd.training.common.enums.LearnDataRegister; import com.kamco.cd.training.common.enums.LearnDataRegister;
import com.kamco.cd.training.common.enums.LearnDataType; import com.kamco.cd.training.common.enums.LearnDataType;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.common.utils.enums.Enums; import com.kamco.cd.training.common.utils.enums.Enums;
import com.kamco.cd.training.common.utils.interfaces.JsonFormatDttm; import com.kamco.cd.training.common.utils.interfaces.JsonFormatDttm;
import io.swagger.v3.oas.annotations.media.Schema; import io.swagger.v3.oas.annotations.media.Schema;
@@ -75,9 +76,16 @@ public class DatasetDto {
} }
public String getTotalSize(Long totalSize) { public String getTotalSize(Long totalSize) {
if (totalSize == null) return "0G"; if (totalSize == null || totalSize <= 0) return "0M";
double giga = totalSize / (1024.0 * 1024 * 1024); double giga = totalSize / (1024.0 * 1024 * 1024);
return String.format("%.2fG", giga);
if (giga >= 1) {
return String.format("%.2fG", giga);
} else {
double mega = totalSize / (1024.0 * 1024);
return String.format("%.2fM", mega);
}
} }
public String getStatus(String status) { public String getStatus(String status) {
@@ -225,9 +233,9 @@ public class DatasetDto {
@Getter @Getter
@Setter @Setter
@NoArgsConstructor @NoArgsConstructor
@JsonInclude(JsonInclude.Include.NON_NULL)
public static class SelectDataSet { public static class SelectDataSet {
private String modelNo; // G1, G2, G3 모델 타입
private Long datasetId; private Long datasetId;
private UUID uuid; private UUID uuid;
private String dataType; private String dataType;
@@ -236,12 +244,16 @@ public class DatasetDto {
private Integer compareYyyy; private Integer compareYyyy;
private Integer targetYyyy; private Integer targetYyyy;
private String memo; private String memo;
private Long classCount; @JsonIgnore private Long classCount;
private Integer buildingCount; private Integer buildingCnt;
private Integer containerCount; private Integer containerCnt;
private String dataTypeName; private String dataTypeName;
private Long wasteCnt;
private Long landCoverCnt;
public SelectDataSet( public SelectDataSet(
String modelNo,
Long datasetId, Long datasetId,
UUID uuid, UUID uuid,
String dataType, String dataType,
@@ -254,15 +266,22 @@ public class DatasetDto {
this.datasetId = datasetId; this.datasetId = datasetId;
this.uuid = uuid; this.uuid = uuid;
this.dataType = dataType; this.dataType = dataType;
this.dataTypeName = getDataTypeName(dataType);
this.title = title; this.title = title;
this.roundNo = roundNo; this.roundNo = roundNo;
this.compareYyyy = compareYyyy; this.compareYyyy = compareYyyy;
this.targetYyyy = targetYyyy; this.targetYyyy = targetYyyy;
this.memo = memo; this.memo = memo;
this.classCount = classCount; this.classCount = classCount;
if (modelNo.equals(ModelType.G2.getId())) {
this.wasteCnt = classCount;
} else if (modelNo.equals(ModelType.G3.getId())) {
this.landCoverCnt = classCount;
}
} }
public SelectDataSet( public SelectDataSet(
String modelNo,
Long datasetId, Long datasetId,
UUID uuid, UUID uuid,
String dataType, String dataType,
@@ -271,8 +290,8 @@ public class DatasetDto {
Integer compareYyyy, Integer compareYyyy,
Integer targetYyyy, Integer targetYyyy,
String memo, String memo,
Integer buildingCount, Integer buildingCnt,
Integer containerCount) { Integer containerCnt) {
this.datasetId = datasetId; this.datasetId = datasetId;
this.uuid = uuid; this.uuid = uuid;
this.dataType = dataType; this.dataType = dataType;
@@ -282,8 +301,8 @@ public class DatasetDto {
this.compareYyyy = compareYyyy; this.compareYyyy = compareYyyy;
this.targetYyyy = targetYyyy; this.targetYyyy = targetYyyy;
this.memo = memo; this.memo = memo;
this.buildingCount = buildingCount; this.buildingCnt = buildingCnt;
this.containerCount = containerCount; this.containerCnt = containerCnt;
} }
public String getDataTypeName(String groupTitleCd) { public String getDataTypeName(String groupTitleCd) {
@@ -296,6 +315,183 @@ public class DatasetDto {
} }
} }
@Schema(name = "SelectTransferDataSet", description = "전이학습 데이터셋 선택 리스트")
@Getter
@Setter
@NoArgsConstructor
public static class SelectTransferDataSet {
private String modelNo; // G1, G2, G3 모델 타입
private Long datasetId;
private UUID uuid;
private String dataType;
private String title;
private Long roundNo;
private Integer compareYyyy;
private Integer targetYyyy;
private String memo;
@JsonIgnore private Long classCount;
private Integer buildingCnt;
private Integer containerCnt;
private String dataTypeName;
private Long wasteCnt;
private Long landCoverCnt;
private String beforeModelNo; // G1, G2, G3 모델 타입
private Long beforeDatasetId;
private UUID beforeUuid;
private String beforeDataType;
private String beforeTitle;
private Long beforeRoundNo;
private Integer beforeCompareYyyy;
private Integer beforeTargetYyyy;
private String beforeMemo;
@JsonIgnore private Long beforeClassCount;
private Integer beforeBuildingCnt;
private Integer beforeContainerCnt;
private String beforeDataTypeName;
private Long beforeWasteCnt;
private Long beforeLandCoverCnt;
public SelectTransferDataSet(
// 현재
String modelNo,
Long datasetId,
UUID uuid,
String dataType,
String title,
Long roundNo,
Integer compareYyyy,
Integer targetYyyy,
String memo,
Long classCount,
// 이전(before)
String beforeModelNo,
Long beforeDatasetId,
UUID beforeUuid,
String beforeDataType,
String beforeTitle,
Long beforeRoundNo,
Integer beforeCompareYyyy,
Integer beforeTargetYyyy,
String beforeMemo,
Long beforeClassCount) {
// 현재
this.modelNo = modelNo;
this.datasetId = datasetId;
this.uuid = uuid;
this.dataType = dataType;
this.dataTypeName = getDataTypeName(dataType);
this.title = title;
this.roundNo = roundNo;
this.compareYyyy = compareYyyy;
this.targetYyyy = targetYyyy;
this.memo = memo;
this.classCount = classCount;
if (modelNo != null && modelNo.equals(ModelType.G2.getId())) {
this.wasteCnt = classCount;
} else if (modelNo != null && modelNo.equals(ModelType.G3.getId())) {
this.landCoverCnt = classCount;
}
// 이전(before)
this.beforeModelNo = beforeModelNo;
this.beforeDatasetId = beforeDatasetId;
this.beforeUuid = beforeUuid;
this.beforeDataType = beforeDataType;
this.beforeDataTypeName = getDataTypeName(beforeDataType);
this.beforeTitle = beforeTitle;
this.beforeRoundNo = beforeRoundNo;
this.beforeCompareYyyy = beforeCompareYyyy;
this.beforeTargetYyyy = beforeTargetYyyy;
this.beforeMemo = beforeMemo;
this.beforeClassCount = beforeClassCount;
if (beforeModelNo != null && beforeModelNo.equals(ModelType.G2.getId())) {
this.beforeWasteCnt = beforeClassCount;
} else if (beforeModelNo != null && beforeModelNo.equals(ModelType.G3.getId())) {
this.beforeLandCoverCnt = beforeClassCount;
}
}
public SelectTransferDataSet(
// 현재
String modelNo,
Long datasetId,
UUID uuid,
String dataType,
String title,
Long roundNo,
Integer compareYyyy,
Integer targetYyyy,
String memo,
Integer buildingCnt,
Integer containerCnt,
// 이전(before)
String beforeModelNo,
Long beforeDatasetId,
UUID beforeUuid,
String beforeDataType,
String beforeTitle,
Long beforeRoundNo,
Integer beforeCompareYyyy,
Integer beforeTargetYyyy,
String beforeMemo,
Integer beforeBuildingCnt,
Integer beforeContainerCnt) {
// 현재
this.modelNo = modelNo;
this.datasetId = datasetId;
this.uuid = uuid;
this.dataType = dataType;
this.dataTypeName = getDataTypeName(dataType);
this.title = title;
this.roundNo = roundNo;
this.compareYyyy = compareYyyy;
this.targetYyyy = targetYyyy;
this.memo = memo;
this.buildingCnt = buildingCnt;
this.containerCnt = containerCnt;
// 이전(before)
this.beforeModelNo = beforeModelNo;
this.beforeDatasetId = beforeDatasetId;
this.beforeUuid = beforeUuid;
this.beforeDataType = beforeDataType;
this.beforeDataTypeName = getDataTypeName(beforeDataType);
this.beforeTitle = beforeTitle;
this.beforeRoundNo = beforeRoundNo;
this.beforeCompareYyyy = beforeCompareYyyy;
this.beforeTargetYyyy = beforeTargetYyyy;
this.beforeMemo = beforeMemo;
this.beforeBuildingCnt = beforeBuildingCnt;
this.beforeContainerCnt = beforeContainerCnt;
}
public String getDataTypeName(String groupTitleCd) {
LearnDataType type = Enums.fromId(LearnDataType.class, groupTitleCd);
return type == null ? null : type.getText();
}
public String getYear() {
return this.compareYyyy + "-" + this.targetYyyy;
}
public String getBeforeYear() {
if (this.beforeCompareYyyy == null || this.beforeTargetYyyy == null) {
return null;
}
return this.beforeCompareYyyy + "-" + this.beforeTargetYyyy;
}
}
@Getter @Getter
@Setter @Setter
@NoArgsConstructor @NoArgsConstructor

View File

@@ -21,20 +21,30 @@ import com.kamco.cd.training.dataset.dto.DatasetObjDto.SearchReq;
import com.kamco.cd.training.postgres.core.DatasetCoreService; import com.kamco.cd.training.postgres.core.DatasetCoreService;
import jakarta.validation.Valid; import jakarta.validation.Valid;
import java.io.IOException; import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files; import java.nio.file.Files;
import java.nio.file.Path; import java.nio.file.Path;
import java.nio.file.Paths; import java.nio.file.Paths;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap; import java.util.HashMap;
import java.util.HashSet;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
import java.util.Set;
import java.util.UUID; import java.util.UUID;
import java.util.stream.Collectors;
import java.util.stream.Stream; import java.util.stream.Stream;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value; import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.InputStreamResource;
import org.springframework.core.io.Resource;
import org.springframework.data.domain.Page; import org.springframework.data.domain.Page;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus; import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional; import org.springframework.transaction.annotation.Transactional;
@@ -50,6 +60,8 @@ public class DatasetService {
private String datasetDir; private String datasetDir;
private static final List<String> LABEL_DIRS = List.of("label-json", "label", "input1", "input2"); private static final List<String> LABEL_DIRS = List.of("label-json", "label", "input1", "input2");
private static final List<String> REQUIRED_DIRS = Arrays.asList("train", "val", "test");
private static final List<String> CHECK_DIRS = List.of("label", "input1", "input2");
/** /**
* 데이터셋 목록 조회 * 데이터셋 목록 조회
@@ -164,9 +176,22 @@ public class DatasetService {
Long datasetUid = null; // master id 값, 등록하면서 가져올 예정 Long datasetUid = null; // master id 값, 등록하면서 가져올 예정
try { try {
// 같은 uid 로 등록한 파일이 있는지 확인
Long existsCnt =
datasetCoreService.findDatasetByUidExistsCnt(addReq.getFileName().replace(".zip", ""));
if (existsCnt > 0) {
return new ResponseObj(ApiResponseCode.DUPLICATE_DATA, "이미 등록된 회차 데이터 파일입니다. 확인 부탁드립니다.");
}
// 압축 해제 // 압축 해제
FIleChecker.unzip(addReq.getFileName(), addReq.getFilePath()); FIleChecker.unzip(addReq.getFileName(), addReq.getFilePath());
// 압축 해제한 폴더 하위에 train,val,test 폴더 모두 존재하는지 확인
validateTrainValTestDirs(addReq.getFilePath() + addReq.getFileName().replace(".zip", ""));
// 압축 해제한 폴더의 갯수 맞는지 log 찍기
validateDirFileCount(addReq.getFilePath() + addReq.getFileName().replace(".zip", ""));
// 해제한 폴더 읽어서 데이터 저장 // 해제한 폴더 읽어서 데이터 저장
List<Map<String, Object>> list = List<Map<String, Object>> list =
getUnzipDatasetFiles( getUnzipDatasetFiles(
@@ -179,6 +204,17 @@ public class DatasetService {
idx++; idx++;
} }
List<Map<String, Object>> valList =
getUnzipDatasetFiles(
addReq.getFilePath() + addReq.getFileName().replace(".zip", ""), "val");
int valIdx = 0;
for (Map<String, Object> valid : valList) {
datasetUid =
this.insertTrainTestData(valid, addReq, valIdx, datasetUid, "val"); // val 데이터 insert
valIdx++;
}
List<Map<String, Object>> testList = List<Map<String, Object>> testList =
getUnzipDatasetFiles( getUnzipDatasetFiles(
addReq.getFilePath() + addReq.getFileName().replace(".zip", ""), "test"); addReq.getFilePath() + addReq.getFileName().replace(".zip", ""), "test");
@@ -285,6 +321,8 @@ public class DatasetService {
if (subDir.equals("train")) { if (subDir.equals("train")) {
datasetCoreService.insertDatasetObj(objRegDto); datasetCoreService.insertDatasetObj(objRegDto);
} else if (subDir.equals("val")) {
datasetCoreService.insertDatasetValObj(objRegDto);
} else { } else {
datasetCoreService.insertDatasetTestObj(objRegDto); datasetCoreService.insertDatasetTestObj(objRegDto);
} }
@@ -303,7 +341,10 @@ public class DatasetService {
Path dir = root.resolve(dirName); Path dir = root.resolve(dirName);
if (!Files.isDirectory(dir)) { if (!Files.isDirectory(dir)) {
throw new IllegalStateException("폴더가 존재하지 않습니다 : " + dir); throw new CustomApiException(
ApiResponseCode.NOT_FOUND_DATA.getId(),
HttpStatus.CONFLICT,
"폴더가 존재하지 않습니다. 업로드 된 파일을 확인하세요. : " + dir);
} }
try (Stream<Path> stream = Files.list(dir)) { try (Stream<Path> stream = Files.list(dir)) {
@@ -356,4 +397,113 @@ public class DatasetService {
public String getFilePathByUUIDPathType(UUID uuid, String pathType) { public String getFilePathByUUIDPathType(UUID uuid, String pathType) {
return datasetCoreService.getFilePathByUUIDPathType(uuid, pathType); return datasetCoreService.getFilePathByUUIDPathType(uuid, pathType);
} }
private String escape(String path) {
// 쉘 커맨드에서 안전하게 사용할 수 있도록 문자열을 작은따옴표로 감싸면서, 내부의 작은따옴표를 이스케이프 처리
return "'" + path.replace("'", "'\"'\"'") + "'";
}
private static String normalizeLinuxPath(String path) {
return path.replace("\\", "/");
}
public ResponseEntity<Resource> getFilePathByFile(String remoteFilePath) {
try {
Path path = Paths.get(remoteFilePath);
InputStream inputStream = Files.newInputStream(path);
InputStreamResource resource =
new InputStreamResource(inputStream) {
@Override
public long contentLength() {
return -1; // 알 수 없으면 -1
}
};
String fileName = Paths.get(remoteFilePath.replace("\\", "/")).getFileName().toString();
return ResponseEntity.ok()
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + fileName + "\"")
.contentType(MediaType.APPLICATION_OCTET_STREAM)
.body(resource);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
/** unzipRootDir: 압축 해제된 폴더 경로 (ex: /data/xxx/myzipname) */
public static void validateTrainValTestDirs(String unzipRootDir) {
Path root = Paths.get(unzipRootDir);
// 루트 폴더 자체 존재 확인
if (!Files.exists(root) || !Files.isDirectory(root)) {
throw new CustomApiException(
ApiResponseCode.NOT_FOUND_DATA.getId(),
HttpStatus.CONFLICT,
"압축 해제 폴더가 존재하지 않습니다: " + unzipRootDir);
}
// 필요한 폴더들 존재/디렉토리 여부 확인
List<String> missing =
REQUIRED_DIRS.stream()
.filter(d -> !Files.isDirectory(root.resolve(d)))
.collect(Collectors.toList());
if (!missing.isEmpty()) {
throw new CustomApiException(
ApiResponseCode.NOT_FOUND_DATA.getId(),
HttpStatus.CONFLICT,
"데이터 폴더 구조가 올바르지 않습니다. 누락된 폴더: "
+ String.join(", ", missing)
+ " (필수: train, val, test)");
}
}
public static void validateDirFileCount(String unzipRootDir) {
Path root = Paths.get(unzipRootDir);
for (String split : REQUIRED_DIRS) {
Path splitPath = root.resolve(split);
Map<String, Long> fileCountMap = new HashMap<>();
for (String subDir : CHECK_DIRS) { // input1, input2, label 폴더만 수행하기
Path subDirPath = splitPath.resolve(subDir);
if (!Files.isDirectory(subDirPath)) {
throw new CustomApiException(
ApiResponseCode.NOT_FOUND_DATA.getId(),
HttpStatus.CONFLICT,
split + " 폴더 하위에 " + subDir + " 폴더가 존재하지 않습니다.");
}
long count;
try (Stream<Path> files = Files.list(subDirPath)) {
count = files.filter(Files::isRegularFile).count();
log.info("dir: " + subDirPath + ", count: " + count);
} catch (IOException e) {
throw new CustomApiException(
ApiResponseCode.NOT_FOUND_DATA.getId(),
HttpStatus.CONFLICT,
split + "/" + subDir + " 파일 개수 확인 중 오류 발생");
}
fileCountMap.put(subDir, count);
}
// 모든 폴더 파일 개수가 동일한지 확인
Set<Long> uniqueCounts = new HashSet<>(fileCountMap.values());
if (uniqueCounts.size() != 1) {
throw new CustomApiException(
ApiResponseCode.NOT_FOUND_DATA.getId(),
HttpStatus.CONFLICT,
split + " 데이터 파일 개수가 일치하지 않습니다. " + fileCountMap.toString());
}
}
}
} }

View File

@@ -1,6 +1,7 @@
package com.kamco.cd.training.hyperparam; package com.kamco.cd.training.hyperparam;
import com.kamco.cd.training.common.dto.HyperParam; import com.kamco.cd.training.common.dto.HyperParam;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.config.api.ApiResponseDto; import com.kamco.cd.training.config.api.ApiResponseDto;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto; import com.kamco.cd.training.hyperparam.dto.HyperParamDto;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto.List; import com.kamco.cd.training.hyperparam.dto.HyperParamDto.List;
@@ -65,7 +66,7 @@ public class HyperParamApiController {
mediaType = "application/json", mediaType = "application/json",
schema = @Schema(implementation = String.class))), schema = @Schema(implementation = String.class))),
@ApiResponse(responseCode = "400", description = "잘못된 요청", content = @Content), @ApiResponse(responseCode = "400", description = "잘못된 요청", content = @Content),
@ApiResponse(responseCode = "422", description = "HPs_0001 수정 불가", content = @Content), @ApiResponse(responseCode = "422", description = "default는 삭제불가", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content) @ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
}) })
@PutMapping("/{uuid}") @PutMapping("/{uuid}")
@@ -96,10 +97,13 @@ public class HyperParamApiController {
String type, String type,
@Parameter(description = "시작일", example = "2026-02-01") @RequestParam(required = false) @Parameter(description = "시작일", example = "2026-02-01") @RequestParam(required = false)
LocalDate startDate, LocalDate startDate,
@Parameter(description = "종료일", example = "2026-02-28") @RequestParam(required = false) @Parameter(description = "종료일", example = "2026-03-31") @RequestParam(required = false)
LocalDate endDate, LocalDate endDate,
@Parameter(description = "버전명", example = "HPs_0001") @RequestParam(required = false) @Parameter(description = "버전명", example = "G1_000019") @RequestParam(required = false)
String hyperVer, String hyperVer,
@Parameter(description = "모델 타입 (G1, G2, G3 중 하나)", example = "G1")
@RequestParam(required = false)
ModelType model,
@Parameter( @Parameter(
description = "정렬", description = "정렬",
example = "createdDttm desc", example = "createdDttm desc",
@@ -124,7 +128,7 @@ public class HyperParamApiController {
searchReq.setSort(sort); searchReq.setSort(sort);
searchReq.setPage(page); searchReq.setPage(page);
searchReq.setSize(size); searchReq.setSize(size);
Page<List> list = hyperParamService.getHyperParamList(searchReq); Page<List> list = hyperParamService.getHyperParamList(model, searchReq);
return ApiResponseDto.ok(list); return ApiResponseDto.ok(list);
} }
@@ -133,12 +137,12 @@ public class HyperParamApiController {
@ApiResponses( @ApiResponses(
value = { value = {
@ApiResponse(responseCode = "200", description = "삭제 성공", content = @Content), @ApiResponse(responseCode = "200", description = "삭제 성공", content = @Content),
@ApiResponse(responseCode = "422", description = "HPs_0001 삭제 불가", content = @Content), @ApiResponse(responseCode = "422", description = "default 삭제 불가", content = @Content),
@ApiResponse(responseCode = "404", description = "하이퍼파라미터를 찾을 수 없음", content = @Content), @ApiResponse(responseCode = "404", description = "하이퍼파라미터를 찾을 수 없음", content = @Content),
}) })
@DeleteMapping("/{uuid}") @DeleteMapping("/{uuid}")
public ApiResponseDto<Void> deleteHyperParam( public ApiResponseDto<Void> deleteHyperParam(
@Parameter(description = "하이퍼파라미터 uuid", example = "c3b5a285-8f68-42af-84f0-e6d09162deb5") @Parameter(description = "하이퍼파라미터 uuid", example = "57fc9170-64c1-4128-aa7b-0657f08d6d10")
@PathVariable @PathVariable
UUID uuid) { UUID uuid) {
hyperParamService.deleteHyperParam(uuid); hyperParamService.deleteHyperParam(uuid);
@@ -160,7 +164,7 @@ public class HyperParamApiController {
}) })
@GetMapping("/{uuid}") @GetMapping("/{uuid}")
public ApiResponseDto<HyperParamDto.Basic> getHyperParam( public ApiResponseDto<HyperParamDto.Basic> getHyperParam(
@Parameter(description = "하이퍼파라미터 uuid", example = "c3b5a285-8f68-42af-84f0-e6d09162deb5") @Parameter(description = "하이퍼파라미터 uuid", example = "57fc9170-64c1-4128-aa7b-0657f08d6d10")
@PathVariable @PathVariable
UUID uuid) { UUID uuid) {
return ApiResponseDto.ok(hyperParamService.getHyperParam(uuid)); return ApiResponseDto.ok(hyperParamService.getHyperParam(uuid));
@@ -179,8 +183,9 @@ public class HyperParamApiController {
@ApiResponse(responseCode = "404", description = "하이퍼파라미터를 찾을 수 없음", content = @Content), @ApiResponse(responseCode = "404", description = "하이퍼파라미터를 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content) @ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
}) })
@GetMapping("/init") @GetMapping("/init/{model}")
public ApiResponseDto<HyperParamDto.Basic> getInitHyperParam() { public ApiResponseDto<HyperParamDto.Basic> getInitHyperParam(@PathVariable ModelType model) {
return ApiResponseDto.ok(hyperParamService.getInitHyperParam());
return ApiResponseDto.ok(hyperParamService.getInitHyperParam(model));
} }
} }

View File

@@ -1,5 +1,6 @@
package com.kamco.cd.training.hyperparam.dto; package com.kamco.cd.training.hyperparam.dto;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.common.utils.enums.CodeExpose; import com.kamco.cd.training.common.utils.enums.CodeExpose;
import com.kamco.cd.training.common.utils.enums.EnumType; import com.kamco.cd.training.common.utils.enums.EnumType;
import com.kamco.cd.training.common.utils.interfaces.JsonFormatDttm; import com.kamco.cd.training.common.utils.interfaces.JsonFormatDttm;
@@ -24,9 +25,12 @@ public class HyperParamDto {
@AllArgsConstructor @AllArgsConstructor
public static class Basic { public static class Basic {
private ModelType model; // 20250212 modeltype추가
private UUID uuid; private UUID uuid;
private String hyperVer; private String hyperVer;
@JsonFormatDttm private ZonedDateTime createdDttm; @JsonFormatDttm private ZonedDateTime createdDttm;
@JsonFormatDttm private ZonedDateTime lastUsedDttm;
private Integer totalUseCnt;
// ------------------------- // -------------------------
// Important // Important
@@ -98,6 +102,8 @@ public class HyperParamDto {
private Integer gpuCnt; private Integer gpuCnt;
private String gpuIds; private String gpuIds;
private Integer masterPort; private Integer masterPort;
private Boolean isDefault;
} }
@Getter @Getter
@@ -106,13 +112,12 @@ public class HyperParamDto {
@AllArgsConstructor @AllArgsConstructor
public static class List { public static class List {
private UUID uuid; private UUID uuid;
private ModelType model;
private String hyperVer; private String hyperVer;
@JsonFormatDttm private ZonedDateTime createDttm; @JsonFormatDttm private ZonedDateTime createDttm;
@JsonFormatDttm private ZonedDateTime lastUsedDttm; @JsonFormatDttm private ZonedDateTime lastUsedDttm;
private Long m1UseCnt; private String memo;
private Long m2UseCnt; private Integer totalUseCnt;
private Long m3UseCnt;
private Long totalCnt;
} }
@Getter @Getter

View File

@@ -1,8 +1,10 @@
package com.kamco.cd.training.hyperparam.service; package com.kamco.cd.training.hyperparam.service;
import com.kamco.cd.training.common.dto.HyperParam; import com.kamco.cd.training.common.dto.HyperParam;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto; import com.kamco.cd.training.hyperparam.dto.HyperParamDto;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto.List; import com.kamco.cd.training.hyperparam.dto.HyperParamDto.List;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto.SearchReq;
import com.kamco.cd.training.postgres.core.HyperParamCoreService; import com.kamco.cd.training.postgres.core.HyperParamCoreService;
import java.util.UUID; import java.util.UUID;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
@@ -20,11 +22,12 @@ public class HyperParamService {
/** /**
* 하이퍼 파라미터 목록 조회 * 하이퍼 파라미터 목록 조회
* *
* @param model
* @param req * @param req
* @return 목록 * @return 목록
*/ */
public Page<List> getHyperParamList(HyperParamDto.SearchReq req) { public Page<List> getHyperParamList(ModelType model, SearchReq req) {
return hyperParamCoreService.findByHyperVerList(req); return hyperParamCoreService.findByHyperVerList(model, req);
} }
/** /**
@@ -59,8 +62,8 @@ public class HyperParamService {
} }
/** 하이퍼파라미터 최적화 설정값 조회 */ /** 하이퍼파라미터 최적화 설정값 조회 */
public HyperParamDto.Basic getInitHyperParam() { public HyperParamDto.Basic getInitHyperParam(ModelType model) {
return hyperParamCoreService.getInitHyperParam(); return hyperParamCoreService.getInitHyperParam(model);
} }
/** /**

View File

@@ -0,0 +1,99 @@
package com.kamco.cd.training.log;
import com.kamco.cd.training.config.api.ApiResponseDto;
import com.kamco.cd.training.log.dto.AuditLogDto;
import com.kamco.cd.training.log.service.AuditLogService;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import java.time.LocalDate;
import lombok.RequiredArgsConstructor;
import org.springframework.data.domain.Page;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
@Tag(name = "감사 로그", description = "감사 로그 관리 API")
@RequiredArgsConstructor
@RestController
@RequestMapping("/api/logs/audit")
public class AuditLogApiController {
private final AuditLogService auditLogService;
@Operation(summary = "일자별 로그 조회")
@GetMapping("/daily")
public ApiResponseDto<Page<AuditLogDto.DailyAuditList>> getDailyLogs(
@RequestParam(required = false) LocalDate startDate,
@RequestParam(required = false) LocalDate endDate,
@RequestParam int page,
@RequestParam(defaultValue = "20") int size) {
AuditLogDto.searchReq searchReq = new AuditLogDto.searchReq(page, size, "created_dttm,desc");
Page<AuditLogDto.DailyAuditList> result =
auditLogService.getLogByDaily(searchReq, startDate, endDate);
return ApiResponseDto.ok(result);
}
@Operation(summary = "일자별 로그 상세")
@GetMapping("/daily/result")
public ApiResponseDto<Page<AuditLogDto.DailyDetail>> getDailyResultLogs(
@RequestParam LocalDate logDate,
@RequestParam int page,
@RequestParam(defaultValue = "20") int size) {
AuditLogDto.searchReq searchReq = new AuditLogDto.searchReq(page, size, "created_dttm,desc");
Page<AuditLogDto.DailyDetail> result = auditLogService.getLogByDailyResult(searchReq, logDate);
return ApiResponseDto.ok(result);
}
@Operation(summary = "메뉴별 로그 조회")
@GetMapping("/menu")
public ApiResponseDto<Page<AuditLogDto.MenuAuditList>> getMenuLogs(
@RequestParam(required = false) String searchValue,
@RequestParam int page,
@RequestParam(defaultValue = "20") int size) {
AuditLogDto.searchReq searchReq = new AuditLogDto.searchReq(page, size, "created_dttm,desc");
Page<AuditLogDto.MenuAuditList> result = auditLogService.getLogByMenu(searchReq, searchValue);
return ApiResponseDto.ok(result);
}
@Operation(summary = "메뉴별 로그 상세")
@GetMapping("/menu/result")
public ApiResponseDto<Page<AuditLogDto.MenuDetail>> getMenuResultLogs(
@RequestParam String menuId,
@RequestParam int page,
@RequestParam(defaultValue = "20") int size) {
AuditLogDto.searchReq searchReq = new AuditLogDto.searchReq(page, size, "created_dttm,desc");
Page<AuditLogDto.MenuDetail> result = auditLogService.getLogByMenuResult(searchReq, menuId);
return ApiResponseDto.ok(result);
}
@Operation(summary = "사용자별 로그 조회")
@GetMapping("/account")
public ApiResponseDto<Page<AuditLogDto.UserAuditList>> getAccountLogs(
@RequestParam(required = false) String searchValue,
@RequestParam int page,
@RequestParam(defaultValue = "20") int size) {
AuditLogDto.searchReq searchReq = new AuditLogDto.searchReq(page, size, "created_dttm,desc");
Page<AuditLogDto.UserAuditList> result =
auditLogService.getLogByAccount(searchReq, searchValue);
return ApiResponseDto.ok(result);
}
@Operation(summary = "사용자별 로그 상세")
@GetMapping("/account/result")
public ApiResponseDto<Page<AuditLogDto.UserDetail>> getAccountResultLogs(
@RequestParam Long userUid,
@RequestParam int page,
@RequestParam(defaultValue = "20") int size) {
AuditLogDto.searchReq searchReq = new AuditLogDto.searchReq(page, size, "created_dttm,desc");
Page<AuditLogDto.UserDetail> result = auditLogService.getLogByAccountResult(searchReq, userUid);
return ApiResponseDto.ok(result);
}
}

View File

@@ -0,0 +1,40 @@
package com.kamco.cd.training.log;
import com.kamco.cd.training.config.api.ApiResponseDto;
import com.kamco.cd.training.log.dto.ErrorLogDto;
import com.kamco.cd.training.log.dto.EventType;
import com.kamco.cd.training.log.service.ErrorLogService;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import java.time.LocalDate;
import lombok.RequiredArgsConstructor;
import org.springframework.data.domain.Page;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
@Tag(name = "에러 로그", description = "에러 로그 관리 API")
@RequiredArgsConstructor
@RestController
@RequestMapping({"/api/logs/system"})
public class ErrorLogApiController {
private final ErrorLogService errorLogService;
@Operation(summary = "에러로그 조회")
@GetMapping("/error")
public ApiResponseDto<Page<ErrorLogDto.Basic>> getErrorLogs(
@RequestParam(required = false) ErrorLogDto.LogErrorLevel logErrorLevel,
@RequestParam(required = false) EventType eventType,
@RequestParam(required = false) LocalDate startDate,
@RequestParam(required = false) LocalDate endDate,
@RequestParam int page,
@RequestParam(defaultValue = "20") int size) {
ErrorLogDto.ErrorSearchReq searchReq =
new ErrorLogDto.ErrorSearchReq(
logErrorLevel, eventType, startDate, endDate, page, size, "created_dttm,desc");
Page<ErrorLogDto.Basic> result = errorLogService.findLogByError(searchReq);
return ApiResponseDto.ok(result);
}
}

View File

@@ -3,7 +3,9 @@ package com.kamco.cd.training.log.dto;
import com.fasterxml.jackson.annotation.JsonIgnore; import com.fasterxml.jackson.annotation.JsonIgnore;
import com.kamco.cd.training.common.utils.interfaces.JsonFormatDttm; import com.kamco.cd.training.common.utils.interfaces.JsonFormatDttm;
import io.swagger.v3.oas.annotations.media.Schema; import io.swagger.v3.oas.annotations.media.Schema;
import java.time.LocalDate;
import java.time.ZonedDateTime; import java.time.ZonedDateTime;
import java.util.UUID;
import lombok.AllArgsConstructor; import lombok.AllArgsConstructor;
import lombok.Getter; import lombok.Getter;
import lombok.NoArgsConstructor; import lombok.NoArgsConstructor;
@@ -58,6 +60,7 @@ public class AuditLogDto {
@Getter @Getter
@AllArgsConstructor @AllArgsConstructor
public static class AuditCommon { public static class AuditCommon {
private int readCount; private int readCount;
private int cudCount; private int cudCount;
private int printCount; private int printCount;
@@ -68,6 +71,7 @@ public class AuditLogDto {
@Schema(name = "DailyAuditList", description = "일자별 목록") @Schema(name = "DailyAuditList", description = "일자별 목록")
@Getter @Getter
public static class DailyAuditList extends AuditCommon { public static class DailyAuditList extends AuditCommon {
private final String baseDate; private final String baseDate;
public DailyAuditList( public DailyAuditList(
@@ -85,6 +89,7 @@ public class AuditLogDto {
@Schema(name = "MenuAuditList", description = "메뉴별 목록") @Schema(name = "MenuAuditList", description = "메뉴별 목록")
@Getter @Getter
public static class MenuAuditList extends AuditCommon { public static class MenuAuditList extends AuditCommon {
private final String menuId; private final String menuId;
private final String menuName; private final String menuName;
@@ -105,6 +110,7 @@ public class AuditLogDto {
@Schema(name = "UserAuditList", description = "사용자별 목록") @Schema(name = "UserAuditList", description = "사용자별 목록")
@Getter @Getter
public static class UserAuditList extends AuditCommon { public static class UserAuditList extends AuditCommon {
private final Long accountId; private final Long accountId;
private final String loginId; private final String loginId;
private final String username; private final String username;
@@ -129,6 +135,7 @@ public class AuditLogDto {
@Getter @Getter
@AllArgsConstructor @AllArgsConstructor
public static class AuditDetail { public static class AuditDetail {
private Long logId; private Long logId;
private EventType eventType; private EventType eventType;
private LogDetail detail; private LogDetail detail;
@@ -137,9 +144,11 @@ public class AuditLogDto {
@Schema(name = "DailyDetail", description = "일자별 로그 상세") @Schema(name = "DailyDetail", description = "일자별 로그 상세")
@Getter @Getter
public static class DailyDetail extends AuditDetail { public static class DailyDetail extends AuditDetail {
private final String userName; private final String userName;
private final String loginId; private final String loginId;
private final String menuName; private final String menuName;
private final String logDateTime;
public DailyDetail( public DailyDetail(
Long logId, Long logId,
@@ -147,17 +156,20 @@ public class AuditLogDto {
String loginId, String loginId,
String menuName, String menuName,
EventType eventType, EventType eventType,
String logDateTime,
LogDetail detail) { LogDetail detail) {
super(logId, eventType, detail); super(logId, eventType, detail);
this.userName = userName; this.userName = userName;
this.loginId = loginId; this.loginId = loginId;
this.menuName = menuName; this.menuName = menuName;
this.logDateTime = logDateTime;
} }
} }
@Schema(name = "MenuDetail", description = "메뉴별 로그 상세") @Schema(name = "MenuDetail", description = "메뉴별 로그 상세")
@Getter @Getter
public static class MenuDetail extends AuditDetail { public static class MenuDetail extends AuditDetail {
private final String logDateTime; private final String logDateTime;
private final String userName; private final String userName;
private final String loginId; private final String loginId;
@@ -179,6 +191,7 @@ public class AuditLogDto {
@Schema(name = "UserDetail", description = "사용자별 로그 상세") @Schema(name = "UserDetail", description = "사용자별 로그 상세")
@Getter @Getter
public static class UserDetail extends AuditDetail { public static class UserDetail extends AuditDetail {
private final String logDateTime; private final String logDateTime;
private final String menuNm; private final String menuNm;
@@ -194,6 +207,7 @@ public class AuditLogDto {
@Setter @Setter
@AllArgsConstructor @AllArgsConstructor
public static class LogDetail { public static class LogDetail {
String serviceName; String serviceName;
String parentMenuName; String parentMenuName;
String menuName; String menuName;
@@ -226,4 +240,26 @@ public class AuditLogDto {
return PageRequest.of(page, size); return PageRequest.of(page, size);
} }
} }
@Getter
@Setter
public static class DownloadReq {
UUID uuid;
LocalDate startDate;
LocalDate endDate;
String searchValue;
String menuId;
String requestUri;
}
@Getter
@Setter
@AllArgsConstructor
public static class DownloadRes {
String name;
String employeeNo;
@JsonFormatDttm ZonedDateTime downloadDttm;
}
} }

View File

@@ -1,5 +1,6 @@
package com.kamco.cd.training.log.dto; package com.kamco.cd.training.log.dto;
import com.kamco.cd.training.common.utils.enums.CodeExpose;
import com.kamco.cd.training.common.utils.enums.EnumType; import com.kamco.cd.training.common.utils.enums.EnumType;
import io.swagger.v3.oas.annotations.media.Schema; import io.swagger.v3.oas.annotations.media.Schema;
import java.time.LocalDate; import java.time.LocalDate;
@@ -77,6 +78,7 @@ public class ErrorLogDto {
} }
} }
@CodeExpose
public enum LogErrorLevel implements EnumType { public enum LogErrorLevel implements EnumType {
WARNING("Warning"), WARNING("Warning"),
ERROR("Error"), ERROR("Error"),

View File

@@ -1,22 +1,35 @@
package com.kamco.cd.training.log.dto; package com.kamco.cd.training.log.dto;
import com.kamco.cd.training.common.utils.enums.CodeExpose;
import com.kamco.cd.training.common.utils.enums.EnumType; import com.kamco.cd.training.common.utils.enums.EnumType;
import lombok.AllArgsConstructor; import lombok.AllArgsConstructor;
import lombok.Getter; import lombok.Getter;
@CodeExpose
@Getter @Getter
@AllArgsConstructor @AllArgsConstructor
public enum EventType implements EnumType { public enum EventType implements EnumType {
CREATE("생성"), LIST("목록"),
READ("조회"), DETAIL("상세"),
UPDATE("수정"), POPUP("팝업"),
DELETE("삭제"), STATUS("상태"),
ADDED("추가"),
MODIFIED("수정"),
REMOVE("삭제"),
DOWNLOAD("다운로드"), DOWNLOAD("다운로드"),
PRINT("출력"), LOGIN("로그인"),
OTHER("기타"); OTHER("기타");
private final String desc; private final String desc;
public static EventType fromName(String name) {
try {
return EventType.valueOf(name.toUpperCase());
} catch (Exception e) {
return OTHER;
}
}
@Override @Override
public String getId() { public String getId() {
return name(); return name();

View File

@@ -1,22 +1,39 @@
package com.kamco.cd.training.model; package com.kamco.cd.training.model;
import com.kamco.cd.training.common.download.RangeDownloadResponder;
import com.kamco.cd.training.config.api.ApiResponseDto; import com.kamco.cd.training.config.api.ApiResponseDto;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto; import com.kamco.cd.training.model.dto.ModelTrainDetailDto;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.MappingDataset; import com.kamco.cd.training.model.dto.ModelTrainDetailDto.MappingDataset;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelBestEpoch;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelFileInfo;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelTestMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelTrainMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelValidationMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.TransferDetailDto; import com.kamco.cd.training.model.dto.ModelTrainDetailDto.TransferDetailDto;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.Basic; import com.kamco.cd.training.model.dto.ModelTrainMngDto.Basic;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.ModelProgressStepDto;
import com.kamco.cd.training.model.service.ModelTrainDetailService; import com.kamco.cd.training.model.service.ModelTrainDetailService;
import com.kamco.cd.training.model.service.ModelTrainMngService;
import io.swagger.v3.oas.annotations.Operation; import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.Parameter; import io.swagger.v3.oas.annotations.Parameter;
import io.swagger.v3.oas.annotations.enums.ParameterIn;
import io.swagger.v3.oas.annotations.media.ArraySchema; import io.swagger.v3.oas.annotations.media.ArraySchema;
import io.swagger.v3.oas.annotations.media.Content; import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema; import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.responses.ApiResponse; import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.responses.ApiResponses; import io.swagger.v3.oas.annotations.responses.ApiResponses;
import io.swagger.v3.oas.annotations.tags.Tag; import io.swagger.v3.oas.annotations.tags.Tag;
import jakarta.servlet.http.HttpServletRequest;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.List; import java.util.List;
import java.util.UUID; import java.util.UUID;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.apache.coyote.BadRequestException;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable; import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMapping;
@@ -28,6 +45,11 @@ import org.springframework.web.bind.annotation.RestController;
@RequestMapping("/api/models") @RequestMapping("/api/models")
public class ModelTrainDetailApiController { public class ModelTrainDetailApiController {
private final ModelTrainDetailService modelTrainDetailService; private final ModelTrainDetailService modelTrainDetailService;
private final ModelTrainMngService modelTrainMngService;
private final RangeDownloadResponder rangeDownloadResponder;
@Value("${train.docker.responseDir}")
private String responseDir;
@Operation(summary = "모델학습관리> 모델관리 > 상세정보탭 > 학습 진행정보", description = "학습 진행정보, 모델학습 정보 API") @Operation(summary = "모델학습관리> 모델관리 > 상세정보탭 > 학습 진행정보", description = "학습 진행정보, 모델학습 정보 API")
@ApiResponses( @ApiResponses(
@@ -112,7 +134,28 @@ public class ModelTrainDetailApiController {
return ApiResponseDto.ok(modelTrainDetailService.getByModelMappingDataset(uuid)); return ApiResponseDto.ok(modelTrainDetailService.getByModelMappingDataset(uuid));
} }
@Operation(summary = "모델관리 > 전이 학습 실행설정 > 모델선택", description = "모델선택 정보 API") // @Operation(summary = "모델관리 > 전이 학습 실행설정 > 모델선택", description = "모델선택 정보 API")
// @ApiResponses(
// value = {
// @ApiResponse(
// responseCode = "200",
// description = "조회 성공",
// content =
// @Content(
// mediaType = "application/json",
// schema = @Schema(implementation = TransferDetailDto.class))),
// @ApiResponse(responseCode = "404", description = "데이터셋을 찾을 수 없음", content = @Content),
// @ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
// })
// @GetMapping("/transfer/detail/{uuid}")
// public ApiResponseDto<TransferDetailDto> getTransferDetail(
// @Parameter(description = "모델 uuid", example = "7fbdff54-ea87-4b02-90d1-955fa2a3457e")
// @PathVariable
// UUID uuid) {
// return ApiResponseDto.ok(modelTrainDetailService.getTransferDetail(uuid));
// }
@Operation(summary = "모델관리 > 모델 상세 > 성능 정보 (Train)", description = "모델 상세 > 성능 정보 (Train) API")
@ApiResponses( @ApiResponses(
value = { value = {
@ApiResponse( @ApiResponse(
@@ -125,11 +168,162 @@ public class ModelTrainDetailApiController {
@ApiResponse(responseCode = "404", description = "데이터셋을 찾을 수 없음", content = @Content), @ApiResponse(responseCode = "404", description = "데이터셋을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content) @ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
}) })
@GetMapping("/transfer/detail/{uuid}") @GetMapping("/metrics/train/{uuid}")
public ApiResponseDto<TransferDetailDto> getTransferDetail( public ApiResponseDto<List<ModelTrainMetrics>> getModelTrainMetricResult(
@Parameter(description = "모델 uuid", example = "7fbdff54-ea87-4b02-90d1-955fa2a3457e") @Parameter(description = "모델 uuid", example = "95cb116c-380a-41c0-98d8-4d1142f15bbf")
@PathVariable @PathVariable
UUID uuid) { UUID uuid) {
return ApiResponseDto.ok(modelTrainDetailService.getTransferDetail(uuid)); return ApiResponseDto.ok(modelTrainDetailService.getModelTrainMetricResult(uuid));
}
@Operation(
summary = "모델관리 > 모델 상세 > 성능 정보 (Validation)",
description = "모델 상세 > 성능 정보 (Validation) API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = TransferDetailDto.class))),
@ApiResponse(responseCode = "404", description = "데이터셋을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/metrics/validation/{uuid}")
public ApiResponseDto<List<ModelValidationMetrics>> getModelValidationMetricResult(
@Parameter(description = "모델 uuid", example = "95cb116c-380a-41c0-98d8-4d1142f15bbf")
@PathVariable
UUID uuid) {
return ApiResponseDto.ok(modelTrainDetailService.getModelValidationMetricResult(uuid));
}
@Operation(summary = "모델관리 > 모델 상세 > 성능 정보 (Test)", description = "모델 상세 > 성능 정보 (Test) API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = TransferDetailDto.class))),
@ApiResponse(responseCode = "404", description = "데이터셋을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/metrics/test/{uuid}")
public ApiResponseDto<List<ModelTestMetrics>> getModelTestMetricResult(
@Parameter(description = "모델 uuid", example = "95cb116c-380a-41c0-98d8-4d1142f15bbf")
@PathVariable
UUID uuid) {
return ApiResponseDto.ok(modelTrainDetailService.getModelTestMetricResult(uuid));
}
@Operation(summary = "모델관리 > 모델 상세 > 성능 정보 (Test)", description = "모델 상세 > 성능 정보 (Test) API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = TransferDetailDto.class))),
@ApiResponse(responseCode = "404", description = "데이터셋을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/best-epoch/{uuid}")
public ApiResponseDto<ModelBestEpoch> getModelTrainBestEpoch(
@Parameter(description = "모델 uuid", example = "95cb116c-380a-41c0-98d8-4d1142f15bbf")
@PathVariable
UUID uuid) {
return ApiResponseDto.ok(modelTrainDetailService.getModelTrainBestEpoch(uuid));
}
@Operation(
summary = "학습데이터 파일 다운로드",
description = "학습데이터 파일 다운로드",
parameters = {
@Parameter(
name = "kamco-download-uuid",
in = ParameterIn.HEADER,
required = true,
description = "다운로드 요청 UUID",
schema =
@Schema(
type = "string",
format = "uuid",
example = "6d8d49dc-0c9d-4124-adc7-b9ca610cc394"))
})
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "학습데이터 zip파일 다운로드",
content =
@Content(
mediaType = "application/octet-stream",
schema = @Schema(type = "string", format = "binary"))),
@ApiResponse(responseCode = "404", description = "파일 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/download/{uuid}")
public ResponseEntity<?> download(@PathVariable UUID uuid, HttpServletRequest request)
throws IOException {
Basic info = modelTrainDetailService.findByModelByUUID(uuid);
Path zipPath =
Paths.get(responseDir)
.resolve(String.valueOf(info.getUuid()))
.resolve(info.getModelVer() + ".zip");
if (!Files.isRegularFile(zipPath)) {
throw new BadRequestException();
}
return rangeDownloadResponder.buildZipResponse(zipPath, info.getModelVer() + ".zip", request);
}
@Operation(summary = "모델관리 > 모델 상세 > 파일 정보", description = "모델 상세 > 파일 정보 API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = TransferDetailDto.class))),
@ApiResponse(responseCode = "404", description = "데이터셋을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/file-info/{uuid}")
public ApiResponseDto<ModelFileInfo> getModelTrainFileInfo(
@Parameter(description = "모델 uuid", example = "95cb116c-380a-41c0-98d8-4d1142f15bbf")
@PathVariable
UUID uuid) {
return ApiResponseDto.ok(modelTrainDetailService.getModelTrainFileInfo(uuid));
}
@Operation(summary = "모델관리 > 모델별 진행 상황", description = "모델관리 > 모델별 진행 상황 API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "조회 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = ModelProgressStepDto.class))),
@ApiResponse(responseCode = "404", description = "데이터셋을 찾을 수 없음", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/progress/{uuid}")
public ApiResponseDto<List<ModelProgressStepDto>> findModelTrainProgressInfo(
@Parameter(description = "모델 uuid", example = "95cb116c-380a-41c0-98d8-4d1142f15bbf")
@PathVariable
UUID uuid) {
return ApiResponseDto.ok(modelTrainDetailService.findModelTrainProgressInfo(uuid));
} }
} }

View File

@@ -1,13 +1,17 @@
package com.kamco.cd.training.model; package com.kamco.cd.training.model;
import com.kamco.cd.training.common.dto.MonitorDto;
import com.kamco.cd.training.common.service.SystemMonitorService;
import com.kamco.cd.training.config.api.ApiResponseDto; import com.kamco.cd.training.config.api.ApiResponseDto;
import com.kamco.cd.training.dataset.dto.DatasetDto; import com.kamco.cd.training.dataset.dto.DatasetDto;
import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetReq; import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetReq;
import com.kamco.cd.training.dataset.dto.DatasetDto.SelectDataSet; import com.kamco.cd.training.dataset.dto.DatasetDto.SelectDataSet;
import com.kamco.cd.training.model.dto.ModelConfigDto; import com.kamco.cd.training.model.dto.ModelConfigDto;
import com.kamco.cd.training.model.dto.ModelTrainMngDto; import com.kamco.cd.training.model.dto.ModelTrainMngDto;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.Basic; import com.kamco.cd.training.model.dto.ModelTrainMngDto.ListDto;
import com.kamco.cd.training.model.service.ModelTrainMngService; import com.kamco.cd.training.model.service.ModelTrainMngService;
import com.kamco.cd.training.train.service.ModelTestMetricsJobService;
import com.kamco.cd.training.train.service.ModelTrainMetricsJobService;
import io.swagger.v3.oas.annotations.Operation; import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.Parameter; import io.swagger.v3.oas.annotations.Parameter;
import io.swagger.v3.oas.annotations.media.Content; import io.swagger.v3.oas.annotations.media.Content;
@@ -16,6 +20,7 @@ import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.responses.ApiResponses; import io.swagger.v3.oas.annotations.responses.ApiResponses;
import io.swagger.v3.oas.annotations.tags.Tag; import io.swagger.v3.oas.annotations.tags.Tag;
import jakarta.validation.Valid; import jakarta.validation.Valid;
import java.io.IOException;
import java.util.List; import java.util.List;
import java.util.UUID; import java.util.UUID;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
@@ -35,6 +40,9 @@ import org.springframework.web.bind.annotation.RestController;
@RequestMapping("/api/models") @RequestMapping("/api/models")
public class ModelTrainMngApiController { public class ModelTrainMngApiController {
private final ModelTrainMngService modelTrainMngService; private final ModelTrainMngService modelTrainMngService;
private final ModelTrainMetricsJobService modelTrainMetricsJobService;
private final ModelTestMetricsJobService modelTestMetricsJobService;
private final SystemMonitorService systemMonitorService;
@Operation(summary = "모델학습 목록 조회", description = "모델학습 목록 조회 API") @Operation(summary = "모델학습 목록 조회", description = "모델학습 목록 조회 API")
@ApiResponses( @ApiResponses(
@@ -50,7 +58,7 @@ public class ModelTrainMngApiController {
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content) @ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
}) })
@GetMapping("/list") @GetMapping("/list")
public ApiResponseDto<Page<Basic>> findByModelList( public ApiResponseDto<Page<ListDto>> findByModelList(
@Parameter( @Parameter(
description = "상태코드", description = "상태코드",
example = "IN_PROGRESS", example = "IN_PROGRESS",
@@ -74,7 +82,7 @@ public class ModelTrainMngApiController {
@ApiResponses( @ApiResponses(
value = { value = {
@ApiResponse(responseCode = "200", description = "삭제 성공", content = @Content), @ApiResponse(responseCode = "200", description = "삭제 성공", content = @Content),
@ApiResponse(responseCode = "409", description = "HPs_0001 삭제 불가", content = @Content) @ApiResponse(responseCode = "409", description = "G1_000001 삭제 불가", content = @Content)
}) })
@DeleteMapping("/{uuid}") @DeleteMapping("/{uuid}")
public ApiResponseDto<Void> deleteModelTrain( public ApiResponseDto<Void> deleteModelTrain(
@@ -92,9 +100,8 @@ public class ModelTrainMngApiController {
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content) @ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
}) })
@PostMapping @PostMapping
public ApiResponseDto<String> createModelTrain(@Valid @RequestBody ModelTrainMngDto.AddReq req) { public ApiResponseDto<UUID> createModelTrain(@Valid @RequestBody ModelTrainMngDto.AddReq req) {
modelTrainMngService.createModelTrain(req); return ApiResponseDto.ok(modelTrainMngService.createModelTrain(req));
return ApiResponseDto.ok("ok");
} }
@Operation(summary = "모델학습 config 정보 조회", description = "모델학습 config 정보 조회 API") @Operation(summary = "모델학습 config 정보 조회", description = "모델학습 config 정보 조회 API")
@@ -150,4 +157,82 @@ public class ModelTrainMngApiController {
req.setDataType(selectType); req.setDataType(selectType);
return ApiResponseDto.ok(modelTrainMngService.getDatasetSelectList(req)); return ApiResponseDto.ok(modelTrainMngService.getDatasetSelectList(req));
} }
@Operation(
summary = "모델학습 1단계/2단계 실행중인 것이 있는지 count",
description = "모델학습 1단계/2단계 실행중인 것이 있는지 count")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "검색 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = Long.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/ing-training-cnt")
public ApiResponseDto<Long> findModelStep1InProgressCnt() {
return ApiResponseDto.ok(modelTrainMngService.findModelStep1InProgressCnt());
}
@Operation(
summary = "스케줄러 findTrainValidMetricCsvFiles",
description = "스케줄러 findTrainValidMetricCsvFiles")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "검색 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = Long.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/schedule-trainvalid")
public ApiResponseDto<Long> findTrainValidMetricCsvFiles() {
modelTrainMetricsJobService.findTrainValidMetricCsvFiles();
return ApiResponseDto.ok(null);
}
@Operation(summary = "스케줄러 findTestMetricCsvFiles", description = "스케줄러 findTestMetricCsvFiles")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "검색 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = Long.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/schedule-test")
public ApiResponseDto<Long> findTestValidMetricCsvFiles() throws IOException {
modelTestMetricsJobService.findTestValidMetricCsvFiles();
return ApiResponseDto.ok(null);
}
@Operation(summary = "학습서버 시스템 사용율 조회", description = "cpu, gpu, memory 사용율 조회")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "검색 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = Long.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping("/monitor")
public ApiResponseDto<MonitorDto> getSystem() throws IOException {
return ApiResponseDto.ok(systemMonitorService.get());
}
} }

View File

@@ -20,4 +20,25 @@ public class ModelConfigDto {
private Float testPercent; private Float testPercent;
private String memo; private String memo;
} }
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class TransferBasic {
private Long configId;
private Long modelId;
private Integer epochCount;
private Float trainPercent;
private Float validationPercent;
private Float testPercent;
private String memo;
private Long beforeConfigId;
private Long beforeModelId;
private Integer beforeEpochCount;
private Float beforeTrainPercent;
private Float beforeValidationPercent;
private Float beforeTestPercent;
private String beforeMemo;
}
} }

View File

@@ -6,7 +6,7 @@ import com.kamco.cd.training.common.enums.TrainStatusType;
import com.kamco.cd.training.common.enums.TrainType; import com.kamco.cd.training.common.enums.TrainType;
import com.kamco.cd.training.common.utils.enums.Enums; import com.kamco.cd.training.common.utils.enums.Enums;
import com.kamco.cd.training.common.utils.interfaces.JsonFormatDttm; import com.kamco.cd.training.common.utils.interfaces.JsonFormatDttm;
import com.kamco.cd.training.dataset.dto.DatasetDto.SelectDataSet; import com.kamco.cd.training.dataset.dto.DatasetDto.SelectTransferDataSet;
import io.swagger.v3.oas.annotations.media.Schema; import io.swagger.v3.oas.annotations.media.Schema;
import java.time.Duration; import java.time.Duration;
import java.time.ZonedDateTime; import java.time.ZonedDateTime;
@@ -35,6 +35,7 @@ public class ModelTrainDetailDto {
@JsonFormatDttm private ZonedDateTime step2EndDttm; @JsonFormatDttm private ZonedDateTime step2EndDttm;
private String statusCd; private String statusCd;
private String trainType; private String trainType;
private UUID beforeUuid;
public String getStatusName() { public String getStatusName() {
if (this.statusCd == null || this.statusCd.isBlank()) return null; if (this.statusCd == null || this.statusCd.isBlank()) return null;
@@ -93,6 +94,29 @@ public class ModelTrainDetailDto {
private Integer batchSize; private Integer batchSize;
} }
@Schema(name = "모델학습관리 전이 하이파라미터", description = "모델학습관리 전이 하이파라미터")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class TransferHyperSummary {
private UUID uuid;
private Long hyperParamId;
private String hyperVer;
private String backbone;
private String inputSize;
private String cropSize;
private Integer batchSize;
private UUID beforeUuid;
private Long beforeHyperParamId;
private String beforeHyperVer;
private String beforeBackbone;
private String beforeInputSize;
private String beforeCropSize;
private Integer beforeBatchSize;
}
@Schema(name = "선택한 데이터셋 목록", description = "선택한 데이터셋 목록") @Schema(name = "선택한 데이터셋 목록", description = "선택한 데이터셋 목록")
@Getter @Getter
@Setter @Setter
@@ -153,8 +177,83 @@ public class ModelTrainDetailDto {
@NoArgsConstructor @NoArgsConstructor
@AllArgsConstructor @AllArgsConstructor
public static class TransferDetailDto { public static class TransferDetailDto {
private ModelConfigDto.Basic etcConfig; private ModelConfigDto.TransferBasic etcConfig;
private HyperSummary modelTrainHyper; private TransferHyperSummary modelTrainHyper;
private List<SelectDataSet> modelTrainDataset; private List<SelectTransferDataSet> modelTrainDataset;
// private List<SelectDataSet> beforeTrainDataset;
}
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class ModelTrainMetrics {
private Integer epoch;
private Long iteration;
private Double loss;
private Double lr;
private Float durationTime;
}
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class ModelValidationMetrics {
private Integer epoch;
private Float aAcc;
private Float mFscore;
private Float mPrecision;
private Float mRecall;
private Float mIou;
private Float mAcc;
private Float changedFscore;
private Float changedPrecision;
private Float changedRecall;
private Float unchangedFscore;
private Float unchangedPrecision;
private Float unchangedRecall;
}
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class ModelTestMetrics {
private String model;
private Long tp;
private Long fp;
private Long fn;
private Float precision;
private Float recall;
private Float f1Score;
private Float accuracy;
private Float iou;
private Long detectionCount;
private Long gtCount;
}
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class ModelBestEpoch {
private Integer epoch;
private Double loss;
private Float f1Score;
private Float precision;
private Float recall;
private Float iou;
private Float accuracy;
}
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class ModelFileInfo {
private Boolean fileExistsYn;
private String fileName;
} }
} }

View File

@@ -40,6 +40,14 @@ public class ModelTrainMngDto {
private String statusCd; private String statusCd;
private String trainType; private String trainType;
private String modelNo; private String modelNo;
private Long currentAttemptId;
private String requestPath;
private String packingState;
private ZonedDateTime packingStrtDttm;
private ZonedDateTime packingEndDttm;
private Long beforeModelId;
public String getStatusName() { public String getStatusName() {
if (this.statusCd == null || this.statusCd.isBlank()) return null; if (this.statusCd == null || this.statusCd.isBlank()) return null;
@@ -59,7 +67,7 @@ public class ModelTrainMngDto {
} }
} }
public String getStep2StatusNAme() { public String getStep2StatusName() {
if (this.step2Status == null || this.step2Status.isBlank()) return null; if (this.step2Status == null || this.step2Status.isBlank()) return null;
try { try {
return TrainStatusType.valueOf(this.step2Status).getText(); // 또는 getName() return TrainStatusType.valueOf(this.step2Status).getText(); // 또는 getName()
@@ -98,6 +106,10 @@ public class ModelTrainMngDto {
public String getStep2Duration() { public String getStep2Duration() {
return formatDuration(this.step2StrtDttm, this.step2EndDttm); return formatDuration(this.step2StrtDttm, this.step2EndDttm);
} }
public String getPackingDuration() {
return formatDuration(this.packingStrtDttm, this.packingEndDttm);
}
} }
@Schema(name = "searchReq", description = "모델학습 관리 목록조회 파라미터") @Schema(name = "searchReq", description = "모델학습 관리 목록조회 파라미터")
@@ -137,6 +149,9 @@ public class ModelTrainMngDto {
@Schema(description = "학습타입 GENERAL(일반), TRANSFER(전이)", example = "GENERAL") @Schema(description = "학습타입 GENERAL(일반), TRANSFER(전이)", example = "GENERAL")
private String trainType; private String trainType;
@Schema(description = "전이학습일때 선택한 모델 id")
private Long beforeModelId;
@NotNull @NotNull
@Schema( @Schema(
description = "하이퍼 파라미터 선택 타입 OPTIMIZED(최적화 파라미터),EXISTING(기존 파라미터),NEW(신규 파라미터)", description = "하이퍼 파라미터 선택 타입 OPTIMIZED(최적화 파라미터),EXISTING(기존 파라미터),NEW(신규 파라미터)",
@@ -151,6 +166,17 @@ public class ModelTrainMngDto {
ModelConfig modelConfig; ModelConfig modelConfig;
} }
@Schema(name = "addReq", description = "모델학습 관리 등록 파라미터")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class UpdateReq {
private String requestPath;
private String responsePath;
}
@Getter @Getter
@Setter @Setter
public static class TrainingDataset { public static class TrainingDataset {
@@ -194,4 +220,111 @@ public class ModelTrainMngDto {
@Schema(description = "메모", example = "메모 입니다.") @Schema(description = "메모", example = "메모 입니다.")
private String memo; private String memo;
} }
@Schema(name = "모델학습관리 목록", description = "모델학습관리 목록")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class ListDto {
private Long id;
private UUID uuid;
private String modelVer;
@JsonFormatDttm private ZonedDateTime startDttm;
@JsonFormatDttm private ZonedDateTime step1StrtDttm;
@JsonFormatDttm private ZonedDateTime step1EndDttm;
@JsonFormatDttm private ZonedDateTime step2StrtDttm;
@JsonFormatDttm private ZonedDateTime step2EndDttm;
private String step1Status;
private String step2Status;
private String statusCd;
private String trainType;
private String modelNo;
private Long currentAttemptId;
private String requestPath;
private String packingState;
private ZonedDateTime packingStrtDttm;
private ZonedDateTime packingEndDttm;
private String memo;
private String userNm;
private UUID beforeUuid;
public String getStatusName() {
if (this.statusCd == null || this.statusCd.isBlank()) return null;
try {
return TrainStatusType.valueOf(this.statusCd).getText(); // 또는 getName()
} catch (IllegalArgumentException e) {
return this.statusCd; // 매핑 못하면 코드 그대로 반환(원하면 null 처리)
}
}
public String getStep1StatusName() {
if (this.step1Status == null || this.step1Status.isBlank()) return null;
try {
return TrainStatusType.valueOf(this.step1Status).getText(); // 또는 getName()
} catch (IllegalArgumentException e) {
return this.step1Status; // 매핑 못하면 코드 그대로 반환(원하면 null 처리)
}
}
public String getStep2StatusName() {
if (this.step2Status == null || this.step2Status.isBlank()) return null;
try {
return TrainStatusType.valueOf(this.step2Status).getText(); // 또는 getName()
} catch (IllegalArgumentException e) {
return this.step2Status; // 매핑 못하면 코드 그대로 반환(원하면 null 처리)
}
}
public String getTrainTypeName() {
if (this.trainType == null || this.trainType.isBlank()) return null;
try {
return TrainType.valueOf(this.trainType).getText(); // 또는 getName()
} catch (IllegalArgumentException e) {
return this.trainType; // 매핑 못하면 코드 그대로 반환(원하면 null 처리)
}
}
private String formatDuration(ZonedDateTime start, ZonedDateTime end) {
if (start == null || end == null) {
return null;
}
long totalSeconds = Math.abs(Duration.between(start, end).getSeconds());
long hours = totalSeconds / 3600;
long minutes = (totalSeconds % 3600) / 60;
long seconds = totalSeconds % 60;
return String.format("%d시간 %d분 %d초", hours, minutes, seconds);
}
public String getStep1Duration() {
return formatDuration(this.step1StrtDttm, this.step1EndDttm);
}
public String getStep2Duration() {
return formatDuration(this.step2StrtDttm, this.step2EndDttm);
}
public String getPackingDuration() {
return formatDuration(this.packingStrtDttm, this.packingEndDttm);
}
}
@Getter
@Builder
@AllArgsConstructor
public static class ModelProgressStepDto {
private int step;
private String status;
@JsonFormatDttm private ZonedDateTime startTime;
@JsonFormatDttm private ZonedDateTime endTime;
private boolean isError;
}
} }

View File

@@ -1,13 +1,21 @@
package com.kamco.cd.training.model.service; package com.kamco.cd.training.model.service;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetReq; import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetReq;
import com.kamco.cd.training.dataset.dto.DatasetDto.SelectDataSet; import com.kamco.cd.training.dataset.dto.DatasetDto.SelectTransferDataSet;
import com.kamco.cd.training.model.dto.ModelConfigDto; import com.kamco.cd.training.model.dto.ModelConfigDto;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.DetailSummary; import com.kamco.cd.training.model.dto.ModelTrainDetailDto.DetailSummary;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.HyperSummary; import com.kamco.cd.training.model.dto.ModelTrainDetailDto.HyperSummary;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.MappingDataset; import com.kamco.cd.training.model.dto.ModelTrainDetailDto.MappingDataset;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelBestEpoch;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelFileInfo;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelTestMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelTrainMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelValidationMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.TransferDetailDto; import com.kamco.cd.training.model.dto.ModelTrainDetailDto.TransferDetailDto;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.TransferHyperSummary;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.Basic; import com.kamco.cd.training.model.dto.ModelTrainMngDto.Basic;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.ModelProgressStepDto;
import com.kamco.cd.training.postgres.core.ModelTrainDetailCoreService; import com.kamco.cd.training.postgres.core.ModelTrainDetailCoreService;
import com.kamco.cd.training.postgres.core.ModelTrainMngCoreService; import com.kamco.cd.training.postgres.core.ModelTrainMngCoreService;
import java.util.ArrayList; import java.util.ArrayList;
@@ -55,15 +63,21 @@ public class ModelTrainDetailService {
return modelTrainDetailCoreService.findByModelByUUID(uuid); return modelTrainDetailCoreService.findByModelByUUID(uuid);
} }
/**
* 전이학습 모델선택 정보
*
* @param uuid
* @return
*/
public TransferDetailDto getTransferDetail(UUID uuid) { public TransferDetailDto getTransferDetail(UUID uuid) {
Basic modelInfo = modelTrainDetailCoreService.findByModelByUUID(uuid); Basic modelInfo = modelTrainDetailCoreService.findByModelByUUID(uuid);
// config 정보 조회 // config 정보 조회
ModelConfigDto.Basic configInfo = mngCoreService.findModelConfigByModelId(uuid); ModelConfigDto.TransferBasic configInfo = mngCoreService.findModelTransferConfigByModelId(uuid);
// 하이파라미터 정보 조회 // 하이파라미터 정보 조회
HyperSummary hyperSummary = modelTrainDetailCoreService.getByModelHyperParamSummary(uuid); TransferHyperSummary hyperSummary = modelTrainDetailCoreService.getTransferHyperSummary(uuid);
List<SelectDataSet> dataSets = new ArrayList<>(); List<SelectTransferDataSet> dataSets = new ArrayList<>();
DatasetReq datasetReq = new DatasetReq(); DatasetReq datasetReq = new DatasetReq();
List<Long> datasetIds = new ArrayList<>(); List<Long> datasetIds = new ArrayList<>();
@@ -74,13 +88,39 @@ public class ModelTrainDetailService {
datasetIds.add(mappingDataset.getDatasetId()); datasetIds.add(mappingDataset.getDatasetId());
} }
datasetReq.setIds(datasetIds); datasetReq.setIds(datasetIds);
datasetReq.setModelNo(modelInfo.getModelNo());
if (modelInfo.getModelNo().equals("G1")) { if (modelInfo.getModelNo().equals(ModelType.G1.getId())) {
dataSets = mngCoreService.getDatasetSelectG1List(datasetReq); dataSets = mngCoreService.getDatasetTransferSelectG1List(modelInfo.getId());
} else { } else {
dataSets = mngCoreService.getDatasetSelectG2G3List(datasetReq); dataSets =
mngCoreService.getDatasetTransferSelectG2G3List(
modelInfo.getId(), modelInfo.getModelNo());
} }
// DatasetReq beforeDatasetReq = new DatasetReq();
// List<Long> beforeDatasetIds = new ArrayList<>();
// List<SelectDataSet> beforeDataSets = new ArrayList<>();
//
// Long beforeModelId = modelInfo.getBeforeModelId();
// if (beforeModelId != null) {
// Basic beforeInfo = modelTrainDetailCoreService.findByModelBeforeId(beforeModelId);
// List<MappingDataset> beforeDatasets =
// modelTrainDetailCoreService.getByModelMappingDataset(beforeInfo.getUuid());
//
// for (MappingDataset before : beforeDatasets) {
// beforeDatasetIds.add(before.getDatasetId());
// }
// beforeDatasetReq.setIds(beforeDatasetIds);
// beforeDatasetReq.setModelNo(modelInfo.getModelNo());
//
// if (beforeInfo.getModelNo().equals(ModelType.G1.getId())) {
// beforeDataSets = mngCoreService.getDatasetSelectG1List(beforeDatasetReq);
// } else {
// beforeDataSets = mngCoreService.getDatasetSelectG2G3List(beforeDatasetReq);
// }
// }
TransferDetailDto transferDetailDto = new TransferDetailDto(); TransferDetailDto transferDetailDto = new TransferDetailDto();
transferDetailDto.setEtcConfig(configInfo); transferDetailDto.setEtcConfig(configInfo);
transferDetailDto.setModelTrainHyper(hyperSummary); transferDetailDto.setModelTrainHyper(hyperSummary);
@@ -88,4 +128,28 @@ public class ModelTrainDetailService {
return transferDetailDto; return transferDetailDto;
} }
public List<ModelTrainMetrics> getModelTrainMetricResult(UUID uuid) {
return modelTrainDetailCoreService.getModelTrainMetricResult(uuid);
}
public List<ModelValidationMetrics> getModelValidationMetricResult(UUID uuid) {
return modelTrainDetailCoreService.getModelValidationMetricResult(uuid);
}
public List<ModelTestMetrics> getModelTestMetricResult(UUID uuid) {
return modelTrainDetailCoreService.getModelTestMetricResult(uuid);
}
public ModelBestEpoch getModelTrainBestEpoch(UUID uuid) {
return modelTrainDetailCoreService.getModelTrainBestEpoch(uuid);
}
public ModelFileInfo getModelTrainFileInfo(UUID uuid) {
return modelTrainDetailCoreService.getModelTrainFileInfo(uuid);
}
public List<ModelProgressStepDto> findModelTrainProgressInfo(UUID uuid) {
return modelTrainDetailCoreService.findModelTrainProgressInfo(uuid);
}
} }

View File

@@ -2,6 +2,9 @@ package com.kamco.cd.training.model.service;
import com.kamco.cd.training.common.dto.HyperParam; import com.kamco.cd.training.common.dto.HyperParam;
import com.kamco.cd.training.common.enums.HyperParamSelectType; import com.kamco.cd.training.common.enums.HyperParamSelectType;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.common.enums.TrainType;
import com.kamco.cd.training.common.exception.CustomApiException;
import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetReq; import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetReq;
import com.kamco.cd.training.dataset.dto.DatasetDto.SelectDataSet; import com.kamco.cd.training.dataset.dto.DatasetDto.SelectDataSet;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto; import com.kamco.cd.training.hyperparam.dto.HyperParamDto;
@@ -10,11 +13,13 @@ import com.kamco.cd.training.model.dto.ModelTrainMngDto;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.SearchReq; import com.kamco.cd.training.model.dto.ModelTrainMngDto.SearchReq;
import com.kamco.cd.training.postgres.core.HyperParamCoreService; import com.kamco.cd.training.postgres.core.HyperParamCoreService;
import com.kamco.cd.training.postgres.core.ModelTrainMngCoreService; import com.kamco.cd.training.postgres.core.ModelTrainMngCoreService;
import com.kamco.cd.training.train.service.TrainJobService;
import java.util.List; import java.util.List;
import java.util.UUID; import java.util.UUID;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.springframework.data.domain.Page; import org.springframework.data.domain.Page;
import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional; import org.springframework.transaction.annotation.Transactional;
@@ -26,6 +31,7 @@ public class ModelTrainMngService {
private final ModelTrainMngCoreService modelTrainMngCoreService; private final ModelTrainMngCoreService modelTrainMngCoreService;
private final HyperParamCoreService hyperParamCoreService; private final HyperParamCoreService hyperParamCoreService;
private final TrainJobService trainJobService;
/** /**
* 모델학습 조회 * 모델학습 조회
@@ -33,7 +39,7 @@ public class ModelTrainMngService {
* @param searchReq 검색 조건 * @param searchReq 검색 조건
* @return 페이징 처리된 모델 목록 * @return 페이징 처리된 모델 목록
*/ */
public Page<ModelTrainMngDto.Basic> getModelList(SearchReq searchReq) { public Page<ModelTrainMngDto.ListDto> getModelList(SearchReq searchReq) {
return modelTrainMngCoreService.findByModelList(searchReq); return modelTrainMngCoreService.findByModelList(searchReq);
} }
@@ -54,10 +60,17 @@ public class ModelTrainMngService {
* @return * @return
*/ */
@Transactional @Transactional
public void createModelTrain(ModelTrainMngDto.AddReq req) { public UUID createModelTrain(ModelTrainMngDto.AddReq req) {
HyperParam hyperParam = req.getHyperParam(); HyperParam hyperParam = req.getHyperParam();
HyperParamDto.Basic hyper = new HyperParamDto.Basic(); HyperParamDto.Basic hyper = new HyperParamDto.Basic();
// 전이 학습은 모델 선택 필수
if (TrainType.TRANSFER.getId().equals(req.getTrainType())) {
if (req.getBeforeModelId() == null) {
throw new CustomApiException("BAD_REQUEST", HttpStatus.BAD_REQUEST, "모델을 선택해 주세요.");
}
}
// 하이파라미터 신규저장 // 하이파라미터 신규저장
if (HyperParamSelectType.NEW.getId().equals(req.getHyperParamType())) { if (HyperParamSelectType.NEW.getId().equals(req.getHyperParamType())) {
// 하이퍼파라미터 등록 // 하이퍼파라미터 등록
@@ -66,7 +79,10 @@ public class ModelTrainMngService {
} }
// 모델학습 테이블 저장 // 모델학습 테이블 저장
Long modelId = modelTrainMngCoreService.saveModel(req); ModelTrainMngDto.Basic modelDto = modelTrainMngCoreService.saveModel(req);
Long modelId = modelDto.getId();
UUID modelUuid = modelDto.getUuid();
// 모델학습 데이터셋 저장 // 모델학습 데이터셋 저장
modelTrainMngCoreService.saveModelDataset(modelId, req); modelTrainMngCoreService.saveModelDataset(modelId, req);
@@ -77,6 +93,10 @@ public class ModelTrainMngService {
// 모델 config 저장 // 모델 config 저장
modelTrainMngCoreService.saveModelConfig(modelId, req.getModelConfig()); modelTrainMngCoreService.saveModelConfig(modelId, req.getModelConfig());
// 데이터셋 임시파일 생성
trainJobService.createTmpFile(modelUuid);
return modelUuid;
} }
/** /**
@@ -96,10 +116,14 @@ public class ModelTrainMngService {
* @return * @return
*/ */
public List<SelectDataSet> getDatasetSelectList(DatasetReq req) { public List<SelectDataSet> getDatasetSelectList(DatasetReq req) {
if (req.getModelNo().equals("G1")) { if (req.getModelNo().equals(ModelType.G1.getId())) {
return modelTrainMngCoreService.getDatasetSelectG1List(req); return modelTrainMngCoreService.getDatasetSelectG1List(req);
} else { } else {
return modelTrainMngCoreService.getDatasetSelectG2G3List(req); return modelTrainMngCoreService.getDatasetSelectG2G3List(req);
} }
} }
public Long findModelStep1InProgressCnt() {
return modelTrainMngCoreService.findModelStep1InProgressCnt();
}
} }

View File

@@ -2,6 +2,7 @@ package com.kamco.cd.training.postgres.core;
import com.kamco.cd.training.common.service.BaseCoreService; import com.kamco.cd.training.common.service.BaseCoreService;
import com.kamco.cd.training.log.dto.AuditLogDto; import com.kamco.cd.training.log.dto.AuditLogDto;
import com.kamco.cd.training.log.dto.AuditLogDto.DownloadReq;
import com.kamco.cd.training.postgres.repository.log.AuditLogRepository; import com.kamco.cd.training.postgres.repository.log.AuditLogRepository;
import java.time.LocalDate; import java.time.LocalDate;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
@@ -45,6 +46,11 @@ public class AuditLogCoreService
return auditLogRepository.findLogByAccount(searchRange, searchValue); return auditLogRepository.findLogByAccount(searchRange, searchValue);
} }
public Page<AuditLogDto.DownloadRes> findLogByAccount(
AuditLogDto.searchReq searchReq, DownloadReq downloadReq) {
return auditLogRepository.findDownloadLog(searchReq, downloadReq);
}
public Page<AuditLogDto.DailyDetail> getLogByDailyResult( public Page<AuditLogDto.DailyDetail> getLogByDailyResult(
AuditLogDto.searchReq searchRange, LocalDate logDate) { AuditLogDto.searchReq searchRange, LocalDate logDate) {
return auditLogRepository.findLogByDailyResult(searchRange, logDate); return auditLogRepository.findLogByDailyResult(searchRange, logDate);

View File

@@ -242,4 +242,12 @@ public class DatasetCoreService
entity.setStatus(LearnDataRegister.COMPLETED.getId()); entity.setStatus(LearnDataRegister.COMPLETED.getId());
} }
public void insertDatasetValObj(DatasetObjRegDto objRegDto) {
datasetObjRepository.insertDatasetValObj(objRegDto);
}
public Long findDatasetByUidExistsCnt(String uid) {
return datasetRepository.findDatasetByUidExistsCnt(uid);
}
} }

View File

@@ -1,10 +1,12 @@
package com.kamco.cd.training.postgres.core; package com.kamco.cd.training.postgres.core;
import com.kamco.cd.training.common.dto.HyperParam; import com.kamco.cd.training.common.dto.HyperParam;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.common.exception.CustomApiException; import com.kamco.cd.training.common.exception.CustomApiException;
import com.kamco.cd.training.common.utils.UserUtil; import com.kamco.cd.training.common.utils.UserUtil;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto; import com.kamco.cd.training.hyperparam.dto.HyperParamDto;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto.Basic; import com.kamco.cd.training.hyperparam.dto.HyperParamDto.Basic;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto.SearchReq;
import com.kamco.cd.training.postgres.entity.ModelHyperParamEntity; import com.kamco.cd.training.postgres.entity.ModelHyperParamEntity;
import com.kamco.cd.training.postgres.repository.hyperparam.HyperParamRepository; import com.kamco.cd.training.postgres.repository.hyperparam.HyperParamRepository;
import java.time.ZonedDateTime; import java.time.ZonedDateTime;
@@ -17,6 +19,7 @@ import org.springframework.stereotype.Service;
@Service @Service
@RequiredArgsConstructor @RequiredArgsConstructor
public class HyperParamCoreService { public class HyperParamCoreService {
private final HyperParamRepository hyperParamRepository; private final HyperParamRepository hyperParamRepository;
private final UserUtil userUtil; private final UserUtil userUtil;
@@ -27,11 +30,10 @@ public class HyperParamCoreService {
* @return 등록된 버전명 * @return 등록된 버전명
*/ */
public Basic createHyperParam(HyperParam createReq) { public Basic createHyperParam(HyperParam createReq) {
String firstVersion = getFirstHyperParamVersion(); String firstVersion = getFirstHyperParamVersion(createReq.getModel());
ModelHyperParamEntity entity = new ModelHyperParamEntity(); ModelHyperParamEntity entity = new ModelHyperParamEntity();
entity.setHyperVer(firstVersion); entity.setHyperVer(firstVersion);
applyHyperParam(entity, createReq); applyHyperParam(entity, createReq);
// user // user
@@ -57,7 +59,7 @@ public class HyperParamCoreService {
.findHyperParamByUuid(uuid) .findHyperParamByUuid(uuid)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND)); .orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
if (entity.getHyperVer().equals("HPs_0001")) { if (entity.getIsDefault()) {
throw new CustomApiException("UNPROCESSABLE_ENTITY_UPDATE", HttpStatus.UNPROCESSABLE_ENTITY); throw new CustomApiException("UNPROCESSABLE_ENTITY_UPDATE", HttpStatus.UNPROCESSABLE_ENTITY);
} }
applyHyperParam(entity, createReq); applyHyperParam(entity, createReq);
@@ -69,11 +71,112 @@ public class HyperParamCoreService {
return entity.getHyperVer(); return entity.getHyperVer();
} }
/**
* 하이퍼파라미터 삭제
*
* @param uuid
*/
public void deleteHyperParam(UUID uuid) {
ModelHyperParamEntity entity =
hyperParamRepository
.findHyperParamByUuid(uuid)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
// if (entity.getHyperVer().equals("HPs_0001")) {
// throw new CustomApiException("UNPROCESSABLE_ENTITY", HttpStatus.UNPROCESSABLE_ENTITY);
// }
// 디폴트면 삭제불가
if (entity.getIsDefault()) {
throw new CustomApiException("UNPROCESSABLE_ENTITY", HttpStatus.UNPROCESSABLE_ENTITY);
}
entity.setDelYn(true);
entity.setUpdatedUid(userUtil.getId());
entity.setUpdatedDttm(ZonedDateTime.now());
}
/**
* 하이퍼파라미터 최적화 설정값 조회
*
* @return
*/
public HyperParamDto.Basic getInitHyperParam(ModelType model) {
ModelHyperParamEntity entity =
hyperParamRepository.getHyperParamByType(model).stream()
.filter(e -> e.getIsDefault() == Boolean.TRUE)
.findFirst()
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
return entity.toDto();
}
/**
* 하이퍼파라미터 상세 조회
*
* @return
*/
public HyperParamDto.Basic getHyperParam(UUID uuid) {
ModelHyperParamEntity entity =
hyperParamRepository
.findHyperParamByUuid(uuid)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
return entity.toDto();
}
/**
* 하이퍼파라미터 목록 조회
*
* @param model
* @param req
* @return
*/
public Page<HyperParamDto.List> findByHyperVerList(ModelType model, SearchReq req) {
return hyperParamRepository.findByHyperVerList(model, req);
}
/**
* 하이퍼파라미터 버전 조회
*
* @param model 모델 타입
* @return ver
*/
public String getFirstHyperParamVersion(ModelType model) {
return hyperParamRepository
.findHyperParamVerByModelType(model)
.map(ModelHyperParamEntity::getHyperVer)
.map(ver -> increase(ver, model))
.orElse(model.name() + "_000001");
}
/**
* 하이퍼 파라미터의 버전을 증가시킨다.
*
* @param hyperVer 현재 버전
* @param modelType 모델 타입
* @return 증가된 버전
*/
private String increase(String hyperVer, ModelType modelType) {
String prefix = modelType.name() + "_";
int num = Integer.parseInt(hyperVer.substring(prefix.length()));
return prefix + String.format("%06d", num + 1);
}
private void applyHyperParam(ModelHyperParamEntity entity, HyperParam src) { private void applyHyperParam(ModelHyperParamEntity entity, HyperParam src) {
ModelType model = src.getModel();
// 하드코딩 모델별로 다른경우 250212 bbn 하드코딩
if (model == ModelType.G3) {
entity.setCropSize("512,512");
} else {
entity.setCropSize("256,256");
}
entity.setCropSize(src.getCropSize());
// Important // Important
entity.setModelType(model); // 20250212 modeltype추가
entity.setBackbone(src.getBackbone()); entity.setBackbone(src.getBackbone());
entity.setInputSize(src.getInputSize()); entity.setInputSize(src.getInputSize());
entity.setCropSize(src.getCropSize());
entity.setBatchSize(src.getBatchSize()); entity.setBatchSize(src.getBatchSize());
// Data // Data
@@ -110,79 +213,4 @@ public class HyperParamCoreService {
// memo // memo
entity.setMemo(src.getMemo()); entity.setMemo(src.getMemo());
} }
/**
* 하이퍼파라미터 삭제
*
* @param uuid
*/
public void deleteHyperParam(UUID uuid) {
ModelHyperParamEntity entity =
hyperParamRepository
.findHyperParamByUuid(uuid)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
if (entity.getHyperVer().equals("HPs_0001")) {
throw new CustomApiException("UNPROCESSABLE_ENTITY", HttpStatus.UNPROCESSABLE_ENTITY);
}
entity.setDelYn(true);
entity.setUpdatedUid(userUtil.getId());
entity.setUpdatedDttm(ZonedDateTime.now());
}
/**
* 하이퍼파라미터 최적화 설정값 조회
*
* @return
*/
public HyperParamDto.Basic getInitHyperParam() {
ModelHyperParamEntity entity =
hyperParamRepository
.findHyperParamByHyperVer("HPs_0001")
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
return entity.toDto();
}
/**
* 하이퍼파라미터 상세 조회
*
* @return
*/
public HyperParamDto.Basic getHyperParam(UUID uuid) {
ModelHyperParamEntity entity =
hyperParamRepository
.findHyperParamByUuid(uuid)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
return entity.toDto();
}
/**
* 하이퍼파라미터 목록 조회
*
* @param req
* @return
*/
public Page<HyperParamDto.List> findByHyperVerList(HyperParamDto.SearchReq req) {
return hyperParamRepository.findByHyperVerList(req);
}
/**
* 하이퍼파라미터 버전 조회
*
* @return ver
*/
public String getFirstHyperParamVersion() {
return hyperParamRepository
.findHyperParamVer()
.map(ModelHyperParamEntity::getHyperVer)
.map(this::increase)
.orElse("HPs_0001");
}
private String increase(String hyperVer) {
String prefix = "HPs_";
int num = Integer.parseInt(hyperVer.substring(prefix.length()));
return prefix + String.format("%04d", num + 1);
}
} }

View File

@@ -0,0 +1,50 @@
package com.kamco.cd.training.postgres.core;
import com.kamco.cd.training.postgres.repository.train.ModelTestMetricsJobRepository;
import com.kamco.cd.training.train.dto.ModelTrainMetricsDto.ModelMetricJsonDto;
import com.kamco.cd.training.train.dto.ModelTrainMetricsDto.ModelTestFileName;
import com.kamco.cd.training.train.dto.ModelTrainMetricsDto.ResponsePathDto;
import java.time.ZonedDateTime;
import java.util.List;
import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
@Service
@RequiredArgsConstructor
public class ModelTestMetricsJobCoreService {
private final ModelTestMetricsJobRepository modelTestMetricsJobRepository;
@Transactional
public void updateModelMetricsTrainSaveYn(Long modelId, String stepNo) {
modelTestMetricsJobRepository.updateModelMetricsTrainSaveYn(modelId, stepNo);
}
// Test 로직 시작
public List<ResponsePathDto> getTestMetricSaveNotYetModelIds() {
return modelTestMetricsJobRepository.getTestMetricSaveNotYetModelIds();
}
public void insertModelMetricsTest(List<Object[]> batchArgs) {
modelTestMetricsJobRepository.insertModelMetricsTest(batchArgs);
}
public ModelMetricJsonDto getTestMetricPackingInfo(Long modelId) {
return modelTestMetricsJobRepository.getTestMetricPackingInfo(modelId);
}
public ModelTestFileName findModelTestFileNames(Long modelId) {
return modelTestMetricsJobRepository.findModelTestFileNames(modelId);
}
@Transactional
public void updatePackingStart(Long modelId, ZonedDateTime now) {
modelTestMetricsJobRepository.updatePackingStart(modelId, now);
}
@Transactional
public void updatePackingEnd(Long modelId, ZonedDateTime now, String failSuccState) {
modelTestMetricsJobRepository.updatePackingEnd(modelId, now, failSuccState);
}
}

View File

@@ -7,7 +7,14 @@ import com.kamco.cd.training.model.dto.ModelConfigDto;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.DetailSummary; import com.kamco.cd.training.model.dto.ModelTrainDetailDto.DetailSummary;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.HyperSummary; import com.kamco.cd.training.model.dto.ModelTrainDetailDto.HyperSummary;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.MappingDataset; import com.kamco.cd.training.model.dto.ModelTrainDetailDto.MappingDataset;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelBestEpoch;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelFileInfo;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelTestMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelTrainMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelValidationMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.TransferHyperSummary;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.Basic; import com.kamco.cd.training.model.dto.ModelTrainMngDto.Basic;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.ModelProgressStepDto;
import com.kamco.cd.training.postgres.entity.ModelMasterEntity; import com.kamco.cd.training.postgres.entity.ModelMasterEntity;
import com.kamco.cd.training.postgres.repository.model.ModelConfigRepository; import com.kamco.cd.training.postgres.repository.model.ModelConfigRepository;
import com.kamco.cd.training.postgres.repository.model.ModelDetailRepository; import com.kamco.cd.training.postgres.repository.model.ModelDetailRepository;
@@ -50,10 +57,20 @@ public class ModelTrainDetailCoreService {
return modelDetailRepository.getModelDetailSummary(uuid); return modelDetailRepository.getModelDetailSummary(uuid);
} }
/**
* 하이퍼 파리미터 요약정보
*
* @param uuid 모델마스터 uuid
* @return
*/
public HyperSummary getByModelHyperParamSummary(UUID uuid) { public HyperSummary getByModelHyperParamSummary(UUID uuid) {
return modelDetailRepository.getByModelHyperParamSummary(uuid); return modelDetailRepository.getByModelHyperParamSummary(uuid);
} }
public TransferHyperSummary getTransferHyperSummary(UUID uuid) {
return modelDetailRepository.getByModelTransferHyperParamSummary(uuid);
}
public List<MappingDataset> getByModelMappingDataset(UUID uuid) { public List<MappingDataset> getByModelMappingDataset(UUID uuid) {
return modelDetailRepository.getByModelMappingDataset(uuid); return modelDetailRepository.getByModelMappingDataset(uuid);
} }
@@ -72,4 +89,33 @@ public class ModelTrainDetailCoreService {
public ModelConfigDto.Basic findModelConfig(Long modelId) { public ModelConfigDto.Basic findModelConfig(Long modelId) {
return modelConfigRepository.findModelConfigByModelId(modelId).orElse(null); return modelConfigRepository.findModelConfigByModelId(modelId).orElse(null);
} }
public List<ModelTrainMetrics> getModelTrainMetricResult(UUID uuid) {
return modelDetailRepository.getModelTrainMetricResult(uuid);
}
public List<ModelValidationMetrics> getModelValidationMetricResult(UUID uuid) {
return modelDetailRepository.getModelValidationMetricResult(uuid);
}
public List<ModelTestMetrics> getModelTestMetricResult(UUID uuid) {
return modelDetailRepository.getModelTestMetricResult(uuid);
}
public ModelBestEpoch getModelTrainBestEpoch(UUID uuid) {
return modelDetailRepository.getModelTrainBestEpoch(uuid);
}
public ModelFileInfo getModelTrainFileInfo(UUID uuid) {
return modelDetailRepository.getModelTrainFileInfo(uuid);
}
public List<ModelProgressStepDto> findModelTrainProgressInfo(UUID uuid) {
return modelDetailRepository.findModelTrainProgressInfo(uuid);
}
public Basic findByModelBeforeId(Long beforeModelId) {
ModelMasterEntity entity = modelDetailRepository.findByModelBeforeId(beforeModelId);
return entity.toDto();
}
} }

View File

@@ -0,0 +1,187 @@
package com.kamco.cd.training.postgres.core;
import com.kamco.cd.training.common.exception.CustomApiException;
import com.kamco.cd.training.postgres.entity.ModelTrainJobEntity;
import com.kamco.cd.training.postgres.repository.train.ModelTrainJobRepository;
import com.kamco.cd.training.train.dto.ModelTrainJobDto;
import java.time.ZonedDateTime;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import lombok.RequiredArgsConstructor;
import lombok.extern.log4j.Log4j2;
import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
@Log4j2
@Service
@RequiredArgsConstructor
@Transactional(readOnly = true)
public class ModelTrainJobCoreService {
private final ModelTrainJobRepository modelTrainJobRepository;
public int findMaxAttemptNo(Long modelId) {
return modelTrainJobRepository.findMaxAttemptNo(modelId);
}
public Optional<ModelTrainJobDto> findLatestByModelId(Long modelId) {
return modelTrainJobRepository.findLatestByModelId(modelId).map(ModelTrainJobEntity::toDto);
}
public Optional<ModelTrainJobDto> findById(Long jobId) {
return modelTrainJobRepository.findById(jobId).map(ModelTrainJobEntity::toDto);
}
/** QUEUED Job 생성 */
@Transactional
public Long createQueuedJob(
Long modelId, int attemptNo, Map<String, Object> paramsJson, ZonedDateTime queuedDttm) {
ModelTrainJobEntity job = new ModelTrainJobEntity();
job.setModelId(modelId);
job.setAttemptNo(attemptNo);
job.setStatusCd("QUEUED");
job.setParamsJson(paramsJson);
job.setQueuedDttm(queuedDttm != null ? queuedDttm : ZonedDateTime.now());
modelTrainJobRepository.save(job);
modelTrainJobRepository.flush();
return job.getId();
}
/** 실행 시작 처리 */
@Transactional
public void markRunning(
Long jobId,
String containerName,
String logPath,
String lockedBy,
Integer totalEpoch,
String jobType) {
ModelTrainJobEntity job =
modelTrainJobRepository
.findById(jobId)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
job.setStatusCd("RUNNING");
job.setContainerName(containerName);
job.setLogPath(logPath);
job.setStartedDttm(ZonedDateTime.now());
job.setLockedDttm(ZonedDateTime.now());
job.setLockedBy(lockedBy);
job.setJobType(jobType);
if (totalEpoch != null) {
job.setTotalEpoch(totalEpoch);
}
}
/**
* 성공 처리
*
* @param jobId
* @param exitCode
*/
@Transactional
public void markSuccess(Long jobId, int exitCode) {
ModelTrainJobEntity job =
modelTrainJobRepository
.findById(jobId)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
job.setStatusCd("SUCCESS");
job.setExitCode(exitCode);
job.setFinishedDttm(ZonedDateTime.now());
}
/**
* 실패 처리
*
* @param jobId
* @param exitCode
* @param errorMessage
*/
@Transactional
public void markFailed(Long jobId, Integer exitCode, String errorMessage) {
ModelTrainJobEntity job =
modelTrainJobRepository
.findById(jobId)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
job.setStatusCd("FAILED");
job.setExitCode(exitCode);
job.setErrorMessage(errorMessage);
job.setFinishedDttm(ZonedDateTime.now());
log.info("[TRAIN JOB FAIL] jobId={}, modelId={}", jobId, errorMessage);
}
/**
* 중단됨 처리
*
* @param jobId
* @param exitCode
* @param errorMessage
*/
@Transactional
public void markPaused(Long jobId, Integer exitCode, String errorMessage) {
ModelTrainJobEntity job =
modelTrainJobRepository
.findById(jobId)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
job.setStatusCd("STOPPED");
job.setExitCode(exitCode);
job.setErrorMessage(errorMessage);
job.setFinishedDttm(ZonedDateTime.now());
log.info("[TRAIN JOB FAIL] jobId={}, modelId={}", jobId, errorMessage);
}
/** 취소 처리 */
@Transactional
public void markCanceled(Long jobId) {
ModelTrainJobEntity job =
modelTrainJobRepository
.findById(jobId)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
job.setStatusCd("STOPPED");
job.setFinishedDttm(ZonedDateTime.now());
}
@Transactional
public void updateEpoch(String containerName, Integer epoch) {
ModelTrainJobEntity job =
modelTrainJobRepository
.findByContainerName(containerName)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
job.setCurrentEpoch(epoch);
if (Objects.equals(job.getTotalEpoch(), epoch)) {}
}
public void insertModelTestTrainingRun(Long modelId, Long jobId, int epoch) {
modelTrainJobRepository.insertModelTestTrainingRun(modelId, jobId, epoch);
}
/**
* 실행중인 학습이 있는지 조회
*
* @return
*/
public List<ModelTrainJobDto> findRunningJobs() {
List<ModelTrainJobEntity> entity = modelTrainJobRepository.findRunningJobs();
if (entity == null || entity.isEmpty()) {
return Collections.emptyList();
}
return entity.stream().map(ModelTrainJobEntity::toDto).toList();
}
}

View File

@@ -0,0 +1,37 @@
package com.kamco.cd.training.postgres.core;
import com.kamco.cd.training.postgres.repository.train.ModelTrainMetricsJobRepository;
import com.kamco.cd.training.train.dto.ModelTrainMetricsDto.ResponsePathDto;
import java.util.List;
import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
@Service
@RequiredArgsConstructor
public class ModelTrainMetricsJobCoreService {
private final ModelTrainMetricsJobRepository modelTrainMetricsJobRepository;
public List<ResponsePathDto> getTrainMetricSaveNotYetModelIds() {
return modelTrainMetricsJobRepository.getTrainMetricSaveNotYetModelIds();
}
public void insertModelMetricsTrain(List<Object[]> batchArgs) {
modelTrainMetricsJobRepository.insertModelMetricsTrain(batchArgs);
}
@Transactional
public void updateModelMetricsTrainSaveYn(Long modelId, String stepNo) {
modelTrainMetricsJobRepository.updateModelMetricsTrainSaveYn(modelId, stepNo);
}
public void insertModelMetricsValidation(List<Object[]> batchArgs) {
modelTrainMetricsJobRepository.insertModelMetricsValidation(batchArgs);
}
@Transactional
public void updateModelSelectedBestEpoch(Long modelId, Integer epoch) {
modelTrainMetricsJobRepository.updateModelSelectedBestEpoch(modelId, epoch);
}
}

View File

@@ -8,9 +8,10 @@ import com.kamco.cd.training.common.exception.CustomApiException;
import com.kamco.cd.training.common.utils.UserUtil; import com.kamco.cd.training.common.utils.UserUtil;
import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetReq; import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetReq;
import com.kamco.cd.training.dataset.dto.DatasetDto.SelectDataSet; import com.kamco.cd.training.dataset.dto.DatasetDto.SelectDataSet;
import com.kamco.cd.training.dataset.dto.DatasetDto.SelectTransferDataSet;
import com.kamco.cd.training.model.dto.ModelConfigDto; import com.kamco.cd.training.model.dto.ModelConfigDto;
import com.kamco.cd.training.model.dto.ModelTrainMngDto; import com.kamco.cd.training.model.dto.ModelTrainMngDto;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.Basic; import com.kamco.cd.training.model.dto.ModelTrainMngDto.ListDto;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.TrainingDataset; import com.kamco.cd.training.model.dto.ModelTrainMngDto.TrainingDataset;
import com.kamco.cd.training.postgres.entity.ModelConfigEntity; import com.kamco.cd.training.postgres.entity.ModelConfigEntity;
import com.kamco.cd.training.postgres.entity.ModelDatasetEntity; import com.kamco.cd.training.postgres.entity.ModelDatasetEntity;
@@ -23,17 +24,23 @@ import com.kamco.cd.training.postgres.repository.model.ModelConfigRepository;
import com.kamco.cd.training.postgres.repository.model.ModelDatasetMappRepository; import com.kamco.cd.training.postgres.repository.model.ModelDatasetMappRepository;
import com.kamco.cd.training.postgres.repository.model.ModelDatasetRepository; import com.kamco.cd.training.postgres.repository.model.ModelDatasetRepository;
import com.kamco.cd.training.postgres.repository.model.ModelMngRepository; import com.kamco.cd.training.postgres.repository.model.ModelMngRepository;
import com.kamco.cd.training.train.dto.ModelTrainLinkDto;
import com.kamco.cd.training.train.dto.TrainRunRequest;
import java.time.ZonedDateTime; import java.time.ZonedDateTime;
import java.util.ArrayList;
import java.util.List; import java.util.List;
import java.util.UUID; import java.util.UUID;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.springframework.data.domain.Page; import org.springframework.data.domain.Page;
import org.springframework.http.HttpStatus; import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;
@Service @Service
@RequiredArgsConstructor @RequiredArgsConstructor
public class ModelTrainMngCoreService { public class ModelTrainMngCoreService {
private final ModelMngRepository modelMngRepository; private final ModelMngRepository modelMngRepository;
private final ModelDatasetRepository modelDatasetRepository; private final ModelDatasetRepository modelDatasetRepository;
private final ModelDatasetMappRepository modelDatasetMapRepository; private final ModelDatasetMappRepository modelDatasetMapRepository;
@@ -48,9 +55,10 @@ public class ModelTrainMngCoreService {
* @param searchReq 검색 조건 * @param searchReq 검색 조건
* @return 페이징 처리된 모델 목록 * @return 페이징 처리된 모델 목록
*/ */
public Page<Basic> findByModelList(ModelTrainMngDto.SearchReq searchReq) { public Page<ListDto> findByModelList(ModelTrainMngDto.SearchReq searchReq) {
Page<ModelMasterEntity> entityPage = modelMngRepository.findByModels(searchReq); // Page<ModelMasterEntity> entityPage = modelMngRepository.findByModels(searchReq);
return entityPage.map(ModelMasterEntity::toDto); // return entityPage.map(ModelMasterEntity::toDto);
return modelMngRepository.findByModels(searchReq);
} }
/** /**
@@ -74,13 +82,19 @@ public class ModelTrainMngCoreService {
* @param addReq * @param addReq
* @return * @return
*/ */
public Long saveModel(ModelTrainMngDto.AddReq addReq) { public ModelTrainMngDto.Basic saveModel(ModelTrainMngDto.AddReq addReq) {
ModelMasterEntity entity = new ModelMasterEntity(); ModelMasterEntity entity = new ModelMasterEntity();
ModelHyperParamEntity hyperParamEntity = new ModelHyperParamEntity(); ModelHyperParamEntity hyperParamEntity = new ModelHyperParamEntity();
// 최적화 파라미터는 HPs_0001 사용 // 최적화 파라미터는 모델 type의 디폴트사용
if (HyperParamSelectType.OPTIMIZED.getId().equals(addReq.getHyperParamType())) { if (HyperParamSelectType.OPTIMIZED.getId().equals(addReq.getHyperParamType())) {
hyperParamEntity = hyperParamRepository.findByHyperVer("HPs_0001").orElse(null); ModelType modelType = ModelType.getValueData(addReq.getModelNo());
hyperParamEntity =
hyperParamRepository.getHyperParamByType(modelType).stream()
.filter(e -> e.getIsDefault() == Boolean.TRUE)
.findFirst()
.orElse(null);
// hyperParamEntity = hyperParamRepository.findByHyperVer("HPs_0001").orElse(null);
} else { } else {
hyperParamEntity = hyperParamEntity =
@@ -90,6 +104,12 @@ public class ModelTrainMngCoreService {
if (hyperParamEntity == null || hyperParamEntity.getHyperVer() == null) { if (hyperParamEntity == null || hyperParamEntity.getHyperVer() == null) {
throw new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND); throw new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND);
} }
// 하이퍼 파라미터 사용 횟수 업데이트
hyperParamEntity.setTotalUseCnt(
hyperParamEntity.getTotalUseCnt() == null ? 1 : hyperParamEntity.getTotalUseCnt() + 1);
// 최근 사용일시 업데이트
hyperParamEntity.setLastUsedDttm(ZonedDateTime.now());
String modelVer = String modelVer =
String.join( String.join(
@@ -98,20 +118,19 @@ public class ModelTrainMngCoreService {
entity.setHyperParamId(hyperParamEntity.getId()); entity.setHyperParamId(hyperParamEntity.getId());
entity.setModelNo(addReq.getModelNo()); entity.setModelNo(addReq.getModelNo());
entity.setTrainType(addReq.getTrainType()); // 일반, 전이 entity.setTrainType(addReq.getTrainType()); // 일반, 전이
entity.setBeforeModelId(addReq.getBeforeModelId());
if (addReq.getIsStart()) { entity.setStatusCd(TrainStatusType.READY.getId());
entity.setModelStep((short) 1); entity.setStep1State(TrainStatusType.READY.getId());
entity.setStatusCd(TrainStatusType.IN_PROGRESS.getId());
entity.setStrtDttm(ZonedDateTime.now());
entity.setStep1StrtDttm(ZonedDateTime.now());
entity.setStep1State(TrainStatusType.IN_PROGRESS.getId());
} else {
entity.setStatusCd(TrainStatusType.READY.getId());
}
entity.setCreatedUid(userUtil.getId()); entity.setCreatedUid(userUtil.getId());
ModelMasterEntity resultEntity = modelMngRepository.save(entity); ModelMasterEntity resultEntity = modelMngRepository.save(entity);
return resultEntity.getId();
ModelTrainMngDto.Basic result = new ModelTrainMngDto.Basic();
result.setId(resultEntity.getId());
result.setUuid(resultEntity.getUuid());
return result;
} }
/** /**
@@ -143,6 +162,23 @@ public class ModelTrainMngCoreService {
modelDatasetRepository.save(datasetEntity); modelDatasetRepository.save(datasetEntity);
} }
/**
* 학습모델 수정
*
* @param modelId
* @param req
*/
public void updateModelMaster(Long modelId, ModelTrainMngDto.UpdateReq req) {
ModelMasterEntity entity =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
// 임시폴더 UID업데이트
if (req.getRequestPath() != null && !req.getRequestPath().isEmpty()) {
entity.setRequestPath(req.getRequestPath());
}
}
/** /**
* 모델 데이터셋 mapping 테이블 저장 * 모델 데이터셋 mapping 테이블 저장
* *
@@ -171,7 +207,10 @@ public class ModelTrainMngCoreService {
ModelConfigEntity entity = new ModelConfigEntity(); ModelConfigEntity entity = new ModelConfigEntity();
modelMasterEntity.setId(modelId); modelMasterEntity.setId(modelId);
entity.setModel(modelMasterEntity); entity.setModel(modelMasterEntity);
entity.setEpochCount(req.getEpochCnt()); entity.setEpochCount(
req.getEpochCnt() < 10
? 10
: req.getEpochCnt()); // 에폭이 10 이하이면 10으로 고정하기. 10 이상 에폭으로 해야 best 에폭 파일이 생성되어 내려옴
entity.setTrainPercent(req.getTrainingCnt()); entity.setTrainPercent(req.getTrainingCnt());
entity.setValidationPercent(req.getValidationCnt()); entity.setValidationPercent(req.getValidationCnt());
entity.setTestPercent(req.getTestCnt()); entity.setTestPercent(req.getTestCnt());
@@ -212,6 +251,20 @@ public class ModelTrainMngCoreService {
} }
} }
/**
* uuid로 model id 조회
*
* @param uuid
* @return
*/
public Long findModelIdByUuid(UUID uuid) {
ModelMasterEntity entity =
modelMngRepository
.findByUuid(uuid)
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
return entity.getId();
}
/** /**
* 모델학습 아이디로 config정보 조회 * 모델학습 아이디로 config정보 조회
* *
@@ -225,6 +278,13 @@ public class ModelTrainMngCoreService {
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND)); .orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
} }
public ModelConfigDto.TransferBasic findModelTransferConfigByModelId(UUID uuid) {
ModelMasterEntity modelEntity = findByUuid(uuid);
return modelConfigRepository
.findModelTransferConfigByModelId(modelEntity.getId())
.orElseThrow(() -> new CustomApiException("NOT_FOUND_DATA", HttpStatus.NOT_FOUND));
}
/** /**
* 데이터셋 G1 목록 * 데이터셋 G1 목록
* *
@@ -235,6 +295,16 @@ public class ModelTrainMngCoreService {
return datasetRepository.getDatasetSelectG1List(req); return datasetRepository.getDatasetSelectG1List(req);
} }
/**
* 전이학습 데이터셋 G1 목록
*
* @param modelId 모델 Id
* @return
*/
public List<SelectTransferDataSet> getDatasetTransferSelectG1List(Long modelId) {
return datasetRepository.getDatasetTransferSelectG1List(modelId);
}
/** /**
* 데이터셋 G2, G3 목록 * 데이터셋 G2, G3 목록
* *
@@ -244,4 +314,324 @@ public class ModelTrainMngCoreService {
public List<SelectDataSet> getDatasetSelectG2G3List(DatasetReq req) { public List<SelectDataSet> getDatasetSelectG2G3List(DatasetReq req) {
return datasetRepository.getDatasetSelectG2G3List(req); return datasetRepository.getDatasetSelectG2G3List(req);
} }
/**
* 전이학습 데이터셋 G2, G3 목록
*
* @param modelId 모델 Id
* @param modelNo G2, G3
* @return
*/
public List<SelectTransferDataSet> getDatasetTransferSelectG2G3List(
Long modelId, String modelNo) {
return datasetRepository.getDatasetTransferSelectG2G3List(modelId, modelNo);
}
/**
* 모델관리 조회
*
* @param id
* @return
*/
public ModelTrainMngDto.Basic findModelById(Long id) {
ModelMasterEntity entity =
modelMngRepository
.findById(id)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + id));
return entity.toDto();
}
/** 마스터를 IN_PROGRESS로 전환하고, 현재 실행 jobId를 연결 - UI/중단/상태조회 모두 currentAttemptId를 기준으로 동작 */
@Transactional
public void markInProgress(Long modelId, Long jobId) {
ModelMasterEntity master =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
master.setStatusCd(TrainStatusType.IN_PROGRESS.getId());
master.setCurrentAttemptId(jobId);
// 필요하면 시작시간도 여기서 찍어줌
modelMngRepository.flush();
}
/** 마지막 에러 메시지 초기화 - 재시작/새 실행 때 이전 에러 흔적 제거 */
@Transactional
public void clearLastError(Long modelId) {
ModelMasterEntity master =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
master.setLastError(null);
modelMngRepository.flush();
}
/** 중단 처리(옵션) - cancel에서 쓰려고 하면 같이 구현 */
@Transactional
public void markStopped(Long modelId) {
ModelMasterEntity master =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
master.setStatusCd(TrainStatusType.STOPPED.getId());
}
/** 완료 처리(옵션) - Worker가 성공 시 호출 */
@Transactional
public void markCompleted(Long modelId) {
ModelMasterEntity master =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
master.setStatusCd(TrainStatusType.COMPLETED.getId());
}
/**
* step 1오류 처리(옵션) - Worker가 실패 시 호출
*
* @param modelId
* @param errorMessage
*/
@Transactional
public void markError(Long modelId, String errorMessage) {
ModelMasterEntity master =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
master.setStatusCd(TrainStatusType.ERROR.getId());
master.setStep1State(TrainStatusType.ERROR.getId());
master.setLastError(errorMessage);
master.setUpdatedUid(userUtil.getId());
master.setUpdatedDttm(ZonedDateTime.now());
}
/**
* step 2오류 처리(옵션) - Worker가 실패 시 호출
*
* @param modelId
* @param errorMessage
*/
@Transactional
public void markStep2Error(Long modelId, String errorMessage) {
ModelMasterEntity master =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
master.setStatusCd(TrainStatusType.ERROR.getId());
master.setStep2State(TrainStatusType.ERROR.getId());
master.setLastError(errorMessage);
master.setUpdatedUid(userUtil.getId());
master.setUpdatedDttm(ZonedDateTime.now());
}
/**
* step1 정지 처리
*
* @param modelId
* @param errorMessage
*/
public void markStep1Stop(Long modelId, String errorMessage) {
ModelMasterEntity master =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
master.setStatusCd(TrainStatusType.STOPPED.getId());
master.setStep1State(TrainStatusType.STOPPED.getId());
master.setLastError(errorMessage);
master.setUpdatedUid(userUtil.getId());
master.setUpdatedDttm(ZonedDateTime.now());
}
/**
* step2 정지 처리
*
* @param modelId
* @param errorMessage
*/
public void markStep2Stop(Long modelId, String errorMessage) {
ModelMasterEntity master =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
master.setStatusCd(TrainStatusType.STOPPED.getId());
master.setStep2State(TrainStatusType.STOPPED.getId());
master.setLastError(errorMessage);
master.setUpdatedUid(userUtil.getId());
master.setUpdatedDttm(ZonedDateTime.now());
}
@Transactional
public void markSuccess(Long modelId) {
ModelMasterEntity master =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
// 모델 상태 완료 처리
master.setStatusCd(TrainStatusType.COMPLETED.getId());
// (선택) 마지막 에러 메시지 비우기
master.setLastError(null);
}
/**
* 학습 실행에 필요한 파라미터 조회
*
* @param modelId
* @return
*/
public TrainRunRequest findTrainRunRequest(Long modelId) {
return modelMngRepository.findTrainRunRequest(modelId);
}
/**
* step1 진행중 처리
*
* @param modelId
* @param jobId
*/
@Transactional(propagation = Propagation.REQUIRES_NEW)
public void markStep1InProgress(Long modelId, Long jobId) {
ModelMasterEntity entity =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
entity.setStatusCd(TrainStatusType.IN_PROGRESS.getId());
entity.setStrtDttm(ZonedDateTime.now());
entity.setStep1StrtDttm(ZonedDateTime.now());
entity.setStep1State(TrainStatusType.IN_PROGRESS.getId());
entity.setCurrentAttemptId(jobId);
entity.setUpdatedDttm(ZonedDateTime.now());
entity.setUpdatedUid(userUtil.getId());
}
/**
* step2 진행중 처리
*
* @param modelId
*/
@Transactional(propagation = Propagation.REQUIRES_NEW)
public void markStep2InProgress(Long modelId, Long jobId) {
ModelMasterEntity entity =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
entity.setStatusCd(TrainStatusType.IN_PROGRESS.getId());
entity.setStep2StrtDttm(ZonedDateTime.now());
entity.setStep2State(TrainStatusType.IN_PROGRESS.getId());
entity.setCurrentAttemptId(jobId);
entity.setUpdatedDttm(ZonedDateTime.now());
entity.setUpdatedUid(userUtil.getId());
}
/**
* step1 완료처리
*
* @param modelId
*/
@Transactional(propagation = Propagation.REQUIRES_NEW)
public void markStep1Success(Long modelId) {
ModelMasterEntity entity =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
entity.setStatusCd(TrainStatusType.COMPLETED.getId());
entity.setStep1State(TrainStatusType.COMPLETED.getId());
entity.setStep1EndDttm(ZonedDateTime.now());
entity.setUpdatedDttm(ZonedDateTime.now());
entity.setUpdatedUid(userUtil.getId());
}
/**
* step2 완료처리
*
* @param modelId
*/
@Transactional(propagation = Propagation.REQUIRES_NEW)
public void markStep2Success(Long modelId) {
ModelMasterEntity entity =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
entity.setStatusCd(TrainStatusType.COMPLETED.getId());
entity.setStep2State(TrainStatusType.COMPLETED.getId());
entity.setStep2EndDttm(ZonedDateTime.now());
entity.setUpdatedDttm(ZonedDateTime.now());
entity.setUpdatedUid(userUtil.getId());
}
public void updateModelMasterBestEpoch(Long modelId, int epoch) {
ModelMasterEntity entity =
modelMngRepository
.findById(modelId)
.orElseThrow(() -> new IllegalArgumentException("Model not found: " + modelId));
entity.setBestEpoch(epoch);
}
/**
* 데이터셋 uid 조회
*
* @param datasetIds
* @return
*/
public List<String> findDatasetUid(List<Long> datasetIds) {
return datasetRepository.findDatasetUid(datasetIds);
}
public List<Long> findModelDatasetMapp(Long modelId) {
List<Long> datasetUids = new ArrayList<>();
List<ModelDatasetMappEntity> entities = modelDatasetMapRepository.findByModelUid(modelId);
for (ModelDatasetMappEntity entity : entities) {
datasetUids.add(entity.getDatasetUid());
}
return datasetUids;
}
public Long findModelStep1InProgressCnt() {
return modelMngRepository.findModelStep1InProgressCnt();
}
/**
* train 링크할 파일 경로
*
* @param modelId
* @return
*/
public List<ModelTrainLinkDto> findDatasetTrainPath(Long modelId) {
return modelDatasetMapRepository.findDatasetTrainPath(modelId);
}
/**
* validation 링크할 파일 경로
*
* @param modelId
* @return
*/
public List<ModelTrainLinkDto> findDatasetValPath(Long modelId) {
return modelDatasetMapRepository.findDatasetValPath(modelId);
}
/**
* test 링크할 파일 경로
*
* @param modelId
* @return
*/
public List<ModelTrainLinkDto> findDatasetTestPath(Long modelId) {
return modelDatasetMapRepository.findDatasetTestPath(modelId);
}
} }

View File

@@ -5,6 +5,7 @@ import com.kamco.cd.training.log.dto.EventStatus;
import com.kamco.cd.training.log.dto.EventType; import com.kamco.cd.training.log.dto.EventType;
import com.kamco.cd.training.postgres.CommonCreateEntity; import com.kamco.cd.training.postgres.CommonCreateEntity;
import jakarta.persistence.*; import jakarta.persistence.*;
import java.util.UUID;
import lombok.AccessLevel; import lombok.AccessLevel;
import lombok.Getter; import lombok.Getter;
import lombok.NoArgsConstructor; import lombok.NoArgsConstructor;
@@ -14,6 +15,7 @@ import lombok.NoArgsConstructor;
@NoArgsConstructor(access = AccessLevel.PROTECTED) @NoArgsConstructor(access = AccessLevel.PROTECTED)
@Table(name = "tb_audit_log") @Table(name = "tb_audit_log")
public class AuditLogEntity extends CommonCreateEntity { public class AuditLogEntity extends CommonCreateEntity {
@Id @Id
@GeneratedValue(strategy = GenerationType.IDENTITY) @GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "audit_log_uid", nullable = false) @Column(name = "audit_log_uid", nullable = false)
@@ -43,6 +45,12 @@ public class AuditLogEntity extends CommonCreateEntity {
@Column(name = "error_log_uid") @Column(name = "error_log_uid")
private Long errorLogUid; private Long errorLogUid;
@Column(name = "download_uuid")
private UUID downloadUuid;
@Column(name = "login_attempt_id")
private String loginAttemptId;
public AuditLogEntity( public AuditLogEntity(
Long userUid, Long userUid,
EventType eventType, EventType eventType,
@@ -51,7 +59,9 @@ public class AuditLogEntity extends CommonCreateEntity {
String ipAddress, String ipAddress,
String requestUri, String requestUri,
String requestBody, String requestBody,
Long errorLogUid) { Long errorLogUid,
UUID downloadUuid,
String loginAttemptId) {
this.userUid = userUid; this.userUid = userUid;
this.eventType = eventType; this.eventType = eventType;
this.eventStatus = eventStatus; this.eventStatus = eventStatus;
@@ -60,6 +70,31 @@ public class AuditLogEntity extends CommonCreateEntity {
this.requestUri = requestUri; this.requestUri = requestUri;
this.requestBody = requestBody; this.requestBody = requestBody;
this.errorLogUid = errorLogUid; this.errorLogUid = errorLogUid;
this.downloadUuid = downloadUuid;
this.loginAttemptId = loginAttemptId;
}
/** 파일 다운로드 이력 생성 */
public static AuditLogEntity forFileDownload(
Long userId,
String requestUri,
String menuUid,
String ip,
int httpStatus,
UUID downloadUuid) {
return new AuditLogEntity(
userId,
EventType.DOWNLOAD, // 이벤트 타입 고정
httpStatus < 400 ? EventStatus.SUCCESS : EventStatus.FAILED, // 성공 여부
menuUid,
ip,
requestUri,
null, // requestBody 없음
null, // errorLogUid 없음
downloadUuid,
null // loginAttemptId 없음
);
} }
public AuditLogDto.Basic toDto() { public AuditLogDto.Basic toDto() {

View File

@@ -0,0 +1,117 @@
package com.kamco.cd.training.postgres.entity;
import com.kamco.cd.training.dataset.dto.DatasetObjDto.Basic;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Size;
import java.time.ZonedDateTime;
import java.util.UUID;
import lombok.Getter;
import lombok.Setter;
import org.hibernate.annotations.ColumnDefault;
import org.hibernate.annotations.JdbcTypeCode;
import org.hibernate.type.SqlTypes;
import org.locationtech.jts.geom.Geometry;
@Getter
@Setter
@Entity
@Table(name = "tb_dataset_val_obj")
public class DatasetValObjEntity {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "obj_id", nullable = false)
private Long objId;
@NotNull
@Column(name = "dataset_uid", nullable = false)
private Long datasetUid;
@Column(name = "target_yyyy")
private Integer targetYyyy;
@Size(max = 255)
@Column(name = "target_class_cd")
private String targetClassCd;
@Column(name = "compare_yyyy")
private Integer compareYyyy;
@Size(max = 255)
@Column(name = "compare_class_cd")
private String compareClassCd;
@Size(max = 255)
@Column(name = "target_path")
private String targetPath;
@Size(max = 255)
@Column(name = "compare_path")
private String comparePath;
@Size(max = 255)
@Column(name = "label_path")
private String labelPath;
@Size(max = 255)
@Column(name = "geojson_path")
private String geojsonPath;
@Size(max = 255)
@Column(name = "map_sheet_num")
private String mapSheetNum;
@ColumnDefault("now()")
@Column(name = "created_dttm")
private ZonedDateTime createdDttm;
@Column(name = "created_uid")
private Long createdUid;
@ColumnDefault("false")
@Column(name = "deleted")
private Boolean deleted;
@Column(name = "uuid")
private UUID uuid;
@Size(max = 32)
@Column(name = "uid")
private String uid;
@JdbcTypeCode(SqlTypes.JSON)
@Column(name = "geo_jsonb", columnDefinition = "jsonb")
private String geoJsonb;
@Column(name = "file_name")
private String fileName;
@Column(name = "geom", columnDefinition = "geometry")
private Geometry geom;
public Basic toDto() {
return new Basic(
this.objId,
this.datasetUid,
this.targetYyyy,
this.targetClassCd,
this.compareYyyy,
this.compareClassCd,
this.targetPath,
this.comparePath,
this.labelPath,
this.geojsonPath,
this.mapSheetNum,
this.createdDttm,
this.createdUid,
this.deleted,
this.uuid,
this.geoJsonb);
}
}

View File

@@ -1,5 +1,6 @@
package com.kamco.cd.training.postgres.entity; package com.kamco.cd.training.postgres.entity;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto; import com.kamco.cd.training.hyperparam.dto.HyperParamDto;
import jakarta.persistence.*; import jakarta.persistence.*;
import jakarta.validation.constraints.NotNull; import jakarta.validation.constraints.NotNull;
@@ -191,10 +192,10 @@ public class ModelHyperParamEntity {
@Column(name = "save_best_rule", nullable = false, length = 10) @Column(name = "save_best_rule", nullable = false, length = 10)
private String saveBestRule = "greater"; private String saveBestRule = "greater";
/** Default: 10 */ /** Default: 1 */
@NotNull @NotNull
@Column(name = "val_interval", nullable = false) @Column(name = "val_interval", nullable = false)
private Integer valInterval = 10; private Integer valInterval = 1;
/** Default: 400 */ /** Default: 400 */
@NotNull @NotNull
@@ -302,20 +303,24 @@ public class ModelHyperParamEntity {
@Column(name = "last_used_dttm") @Column(name = "last_used_dttm")
private ZonedDateTime lastUsedDttm; private ZonedDateTime lastUsedDttm;
@Column(name = "m1_use_cnt") @Column(name = "model_type")
private Long m1UseCnt = 0L; @Enumerated(EnumType.STRING)
private ModelType modelType;
@Column(name = "m2_use_cnt") @Column(name = "default_param")
private Long m2UseCnt = 0L; private Boolean isDefault = false;
@Column(name = "m3_use_cnt") @Column(name = "total_use_cnt")
private Long m3UseCnt = 0L; private Integer totalUseCnt = 0;
public HyperParamDto.Basic toDto() { public HyperParamDto.Basic toDto() {
return new HyperParamDto.Basic( return new HyperParamDto.Basic(
this.modelType,
this.uuid, this.uuid,
this.hyperVer, this.hyperVer,
this.createdDttm, this.createdDttm,
this.lastUsedDttm,
this.totalUseCnt,
// ------------------------- // -------------------------
// Important // Important
// ------------------------- // -------------------------
@@ -385,6 +390,7 @@ public class ModelHyperParamEntity {
// ------------------------- // -------------------------
this.gpuCnt, this.gpuCnt,
this.gpuIds, this.gpuIds,
this.masterPort); this.masterPort,
this.isDefault);
} }
} }

View File

@@ -88,6 +88,39 @@ public class ModelMasterEntity {
@Column(name = "train_type") @Column(name = "train_type")
private String trainType; private String trainType;
@Column(name = "before_model_id")
private Long beforeModelId;
@Column(name = "step1_metric_save_yn")
private Boolean step1MetricSaveYn;
@Column(name = "step2_metric_save_yn")
private Boolean step2MetricSaveYn;
@Column(name = "current_attempt_id")
private Long currentAttemptId;
@Column(name = "last_error")
private String lastError;
@Column(name = "best_epoch")
private Integer bestEpoch;
@Column(name = "request_path")
private String requestPath;
@Column(name = "response_path")
private String responsePath;
@Column(name = "packing_state")
private String packingState;
@Column(name = "packing_strt_dttm")
private ZonedDateTime packingStrtDttm;
@Column(name = "packing_end_dttm")
private ZonedDateTime packingEndDttm;
public ModelTrainMngDto.Basic toDto() { public ModelTrainMngDto.Basic toDto() {
return new ModelTrainMngDto.Basic( return new ModelTrainMngDto.Basic(
this.id, this.id,
@@ -102,6 +135,12 @@ public class ModelMasterEntity {
this.step2State, this.step2State,
this.statusCd, this.statusCd,
this.trainType, this.trainType,
this.modelNo); this.modelNo,
this.currentAttemptId,
this.requestPath,
this.packingState,
this.packingStrtDttm,
this.packingEndDttm,
this.beforeModelId);
} }
} }

View File

@@ -19,8 +19,8 @@ import org.hibernate.annotations.ColumnDefault;
@Getter @Getter
@Setter @Setter
@Entity @Entity
@Table(name = "tb_model_matrics_test") @Table(name = "tb_model_metrics_test")
public class ModelMatricsTestEntity { public class ModelMetricsTestEntity {
@Id @Id
@GeneratedValue(strategy = GenerationType.IDENTITY) @GeneratedValue(strategy = GenerationType.IDENTITY)
@@ -45,9 +45,6 @@ public class ModelMatricsTestEntity {
@Column(name = "fn") @Column(name = "fn")
private Long fn; private Long fn;
@Column(name = "tn")
private Long tn;
@Column(name = "precisions") @Column(name = "precisions")
private Float precisions; private Float precisions;
@@ -63,8 +60,11 @@ public class ModelMatricsTestEntity {
@Column(name = "iou") @Column(name = "iou")
private Float iou; private Float iou;
@Column(name = "processed_images") @Column(name = "detection_count")
private Long processedImages; private Long detectionCount;
@Column(name = "gt_count")
private Long gtCount;
@ColumnDefault("now()") @ColumnDefault("now()")
@Column(name = "created_dttm") @Column(name = "created_dttm")

View File

@@ -18,8 +18,8 @@ import org.hibernate.annotations.ColumnDefault;
@Getter @Getter
@Setter @Setter
@Entity @Entity
@Table(name = "tb_model_matrics_train") @Table(name = "tb_model_metrics_train")
public class ModelMatricsTrainEntity { public class ModelMetricsTrainEntity {
@Id @Id
@GeneratedValue(strategy = GenerationType.IDENTITY) @GeneratedValue(strategy = GenerationType.IDENTITY)

View File

@@ -18,8 +18,8 @@ import org.hibernate.annotations.ColumnDefault;
@Getter @Getter
@Setter @Setter
@Entity @Entity
@Table(name = "tb_model_matrics_validation") @Table(name = "tb_model_metrics_validation")
public class ModelMatricsValidationEntity { public class ModelMetricsValidationEntity {
@Id @Id
@GeneratedValue(strategy = GenerationType.IDENTITY) @GeneratedValue(strategy = GenerationType.IDENTITY)

View File

@@ -0,0 +1,42 @@
package com.kamco.cd.training.postgres.entity;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import jakarta.validation.constraints.NotNull;
import java.time.OffsetDateTime;
import lombok.Getter;
import lombok.Setter;
import org.hibernate.annotations.ColumnDefault;
@Getter
@Setter
@Entity
@Table(name = "tb_model_test_training_run")
public class ModelTestTrainingRunEntity {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "tsr_id", nullable = false)
private Long id;
@NotNull
@Column(name = "model_id", nullable = false)
private Long modelId;
@Column(name = "attempt_no")
private Integer attemptNo;
@Column(name = "job_id")
private Long jobId;
@Column(name = "epoch")
private Integer epoch;
@ColumnDefault("now()")
@Column(name = "created_dttm")
private OffsetDateTime createdDttm;
}

View File

@@ -0,0 +1,105 @@
package com.kamco.cd.training.postgres.entity;
import com.kamco.cd.training.train.dto.ModelTrainJobDto;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Size;
import java.time.ZonedDateTime;
import java.util.Map;
import lombok.Getter;
import lombok.Setter;
import org.hibernate.annotations.ColumnDefault;
import org.hibernate.annotations.JdbcTypeCode;
import org.hibernate.type.SqlTypes;
@Getter
@Setter
@Entity
@Table(name = "tb_model_train_job")
public class ModelTrainJobEntity {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "id", nullable = false)
private Long id;
@NotNull
@Column(name = "model_id", nullable = false)
private Long modelId;
@NotNull
@Column(name = "attempt_no", nullable = false)
private Integer attemptNo;
@Size(max = 30)
@NotNull
@Column(name = "status_cd", nullable = false, length = 30)
private String statusCd;
@NotNull
@Column(name = "params_json", nullable = false)
@JdbcTypeCode(SqlTypes.JSON)
private Map<String, Object> paramsJson;
@Size(max = 200)
@Column(name = "container_name", length = 200)
private String containerName;
@Size(max = 500)
@Column(name = "log_path", length = 500)
private String logPath;
@Column(name = "exit_code")
private Integer exitCode;
@Column(name = "error_message", columnDefinition = "TEXT")
private String errorMessage;
@ColumnDefault("now()")
@Column(name = "queued_dttm")
private ZonedDateTime queuedDttm;
@Column(name = "started_dttm")
private ZonedDateTime startedDttm;
@Column(name = "finished_dttm")
private ZonedDateTime finishedDttm;
@Column(name = "locked_dttm")
private ZonedDateTime lockedDttm;
@Size(max = 100)
@Column(name = "locked_by", length = 100)
private String lockedBy;
@Column(name = "total_epoch")
private Integer totalEpoch;
@Column(name = "current_epoch")
private Integer currentEpoch;
@Column(name = "job_type")
private String jobType;
public ModelTrainJobDto toDto() {
return new ModelTrainJobDto(
this.id,
this.modelId,
this.attemptNo,
this.statusCd,
this.exitCode,
this.errorMessage,
this.containerName,
this.paramsJson,
this.queuedDttm,
this.startedDttm,
this.finishedDttm,
this.totalEpoch,
this.currentEpoch);
}
}

View File

@@ -22,4 +22,6 @@ public interface DatasetObjRepositoryCustom {
String getFilePathByUUIDPathType(UUID uuid, String pathType); String getFilePathByUUIDPathType(UUID uuid, String pathType);
void insertDatasetTestObj(DatasetObjRegDto objRegDto); void insertDatasetTestObj(DatasetObjRegDto objRegDto);
void insertDatasetValObj(DatasetObjRegDto objRegDto);
} }

View File

@@ -97,6 +97,49 @@ public class DatasetObjRepositoryImpl implements DatasetObjRepositoryCustom {
} }
} }
@Override
public void insertDatasetValObj(DatasetObjRegDto objRegDto) {
ObjectMapper objectMapper = new ObjectMapper();
String json;
String geometryJson;
try {
json = objectMapper.writeValueAsString(objRegDto.getGeojson());
geometryJson =
objectMapper.writeValueAsString(
objRegDto.getGeojson().path("features").get(0).path("geometry"));
} catch (JsonProcessingException e) {
throw new RuntimeException(e);
}
try {
em.createNativeQuery(
"""
insert into tb_dataset_val_obj
(dataset_uid, target_yyyy, target_class_cd,
compare_yyyy, compare_class_cd,
target_path, compare_path, label_path, geo_jsonb, map_sheet_num, file_name, geom, geojson_path)
values
(?, ?, ?, ?, ?, ?, ?, ?, cast(? as jsonb), ?, ?, ST_SetSRID(ST_GeomFromGeoJSON(?), 5186), ?)
""")
.setParameter(1, objRegDto.getDatasetUid())
.setParameter(2, objRegDto.getTargetYyyy())
.setParameter(3, objRegDto.getTargetClassCd())
.setParameter(4, objRegDto.getCompareYyyy())
.setParameter(5, objRegDto.getCompareClassCd())
.setParameter(6, objRegDto.getTargetPath())
.setParameter(7, objRegDto.getComparePath())
.setParameter(8, objRegDto.getLabelPath())
.setParameter(9, json)
.setParameter(10, objRegDto.getMapSheetNum())
.setParameter(11, objRegDto.getFileName())
.setParameter(12, geometryJson)
.setParameter(13, objRegDto.getGeojsonPath())
.executeUpdate();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
@Override @Override
public Page<DatasetObjEntity> searchDatasetObjectList(SearchReq searchReq) { public Page<DatasetObjEntity> searchDatasetObjectList(SearchReq searchReq) {
Pageable pageable = searchReq.toPageable(); Pageable pageable = searchReq.toPageable();

View File

@@ -4,6 +4,7 @@ import com.kamco.cd.training.dataset.dto.DatasetDto;
import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetMngRegDto; import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetMngRegDto;
import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetReq; import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetReq;
import com.kamco.cd.training.dataset.dto.DatasetDto.SelectDataSet; import com.kamco.cd.training.dataset.dto.DatasetDto.SelectDataSet;
import com.kamco.cd.training.dataset.dto.DatasetDto.SelectTransferDataSet;
import com.kamco.cd.training.postgres.entity.DatasetEntity; import com.kamco.cd.training.postgres.entity.DatasetEntity;
import java.util.List; import java.util.List;
import java.util.Optional; import java.util.Optional;
@@ -17,9 +18,17 @@ public interface DatasetRepositoryCustom {
List<SelectDataSet> getDatasetSelectG1List(DatasetReq req); List<SelectDataSet> getDatasetSelectG1List(DatasetReq req);
public List<SelectTransferDataSet> getDatasetTransferSelectG1List(Long modelId);
public List<SelectTransferDataSet> getDatasetTransferSelectG2G3List(Long modelId, String modelNo);
List<SelectDataSet> getDatasetSelectG2G3List(DatasetReq req); List<SelectDataSet> getDatasetSelectG2G3List(DatasetReq req);
Long getDatasetMaxStage(int compareYyyy, int targetYyyy); Long getDatasetMaxStage(int compareYyyy, int targetYyyy);
Long insertDatasetMngData(DatasetMngRegDto mngRegDto); Long insertDatasetMngData(DatasetMngRegDto mngRegDto);
List<String> findDatasetUid(List<Long> datasetIds);
Long findDatasetByUidExistsCnt(String uid);
} }

View File

@@ -1,17 +1,24 @@
package com.kamco.cd.training.postgres.repository.dataset; package com.kamco.cd.training.postgres.repository.dataset;
import static com.kamco.cd.training.postgres.entity.QDatasetObjEntity.datasetObjEntity; import static com.kamco.cd.training.postgres.entity.QDatasetObjEntity.datasetObjEntity;
import static com.kamco.cd.training.postgres.entity.QModelDatasetMappEntity.modelDatasetMappEntity;
import static com.kamco.cd.training.postgres.entity.QModelMasterEntity.modelMasterEntity;
import com.kamco.cd.training.common.enums.ModelType; import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetMngRegDto; import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetMngRegDto;
import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetReq; import com.kamco.cd.training.dataset.dto.DatasetDto.DatasetReq;
import com.kamco.cd.training.dataset.dto.DatasetDto.SearchReq; import com.kamco.cd.training.dataset.dto.DatasetDto.SearchReq;
import com.kamco.cd.training.dataset.dto.DatasetDto.SelectDataSet; import com.kamco.cd.training.dataset.dto.DatasetDto.SelectDataSet;
import com.kamco.cd.training.dataset.dto.DatasetDto.SelectTransferDataSet;
import com.kamco.cd.training.postgres.entity.DatasetEntity; import com.kamco.cd.training.postgres.entity.DatasetEntity;
import com.kamco.cd.training.postgres.entity.QDatasetEntity; import com.kamco.cd.training.postgres.entity.QDatasetEntity;
import com.kamco.cd.training.postgres.entity.QDatasetObjEntity;
import com.kamco.cd.training.postgres.entity.QModelDatasetMappEntity;
import com.kamco.cd.training.postgres.entity.QModelMasterEntity;
import com.querydsl.core.BooleanBuilder; import com.querydsl.core.BooleanBuilder;
import com.querydsl.core.types.Projections; import com.querydsl.core.types.Projections;
import com.querydsl.core.types.dsl.CaseBuilder; import com.querydsl.core.types.dsl.CaseBuilder;
import com.querydsl.core.types.dsl.Expressions;
import com.querydsl.core.types.dsl.NumberExpression; import com.querydsl.core.types.dsl.NumberExpression;
import com.querydsl.jpa.impl.JPAQueryFactory; import com.querydsl.jpa.impl.JPAQueryFactory;
import java.util.List; import java.util.List;
@@ -66,7 +73,11 @@ public class DatasetRepositoryImpl implements DatasetRepositoryCustom {
// Count 쿼리 별도 실행 (null safe handling) // Count 쿼리 별도 실행 (null safe handling)
long total = long total =
Optional.ofNullable( Optional.ofNullable(
queryFactory.select(dataset.count()).from(dataset).where(builder).fetchOne()) queryFactory
.select(dataset.count())
.from(dataset)
.where(builder.and(dataset.deleted.isFalse()))
.fetchOne())
.orElse(0L); .orElse(0L);
return new PageImpl<>(content, pageable, total); return new PageImpl<>(content, pageable, total);
@@ -87,6 +98,8 @@ public class DatasetRepositoryImpl implements DatasetRepositoryCustom {
BooleanBuilder builder = new BooleanBuilder(); BooleanBuilder builder = new BooleanBuilder();
builder.and(dataset.deleted.isFalse());
if (StringUtils.isNotBlank(req.getDataType()) && !"CURRENT".equals(req.getDataType())) { if (StringUtils.isNotBlank(req.getDataType()) && !"CURRENT".equals(req.getDataType())) {
builder.and(dataset.dataType.eq(req.getDataType())); builder.and(dataset.dataType.eq(req.getDataType()));
} }
@@ -103,6 +116,7 @@ public class DatasetRepositoryImpl implements DatasetRepositoryCustom {
.select( .select(
Projections.constructor( Projections.constructor(
SelectDataSet.class, SelectDataSet.class,
Expressions.constant(req.getModelNo()),
dataset.id, dataset.id,
dataset.uuid, dataset.uuid,
dataset.dataType, dataset.dataType,
@@ -136,10 +150,108 @@ public class DatasetRepositoryImpl implements DatasetRepositoryCustom {
.fetch(); .fetch();
} }
@Override
public List<SelectTransferDataSet> getDatasetTransferSelectG1List(Long modelId) {
QModelMasterEntity beforeMaster = new QModelMasterEntity("beforeMaster");
QModelDatasetMappEntity beforeMapp = new QModelDatasetMappEntity("beforeMapp");
QDatasetEntity beforeDataset = new QDatasetEntity("beforeDataset");
QDatasetObjEntity beforeObj = new QDatasetObjEntity("beforeObj");
return queryFactory
.select(
Projections.constructor(
SelectTransferDataSet.class,
// ===== 현재 =====
modelMasterEntity.modelNo,
dataset.id,
dataset.uuid,
dataset.dataType,
dataset.title,
dataset.roundNo,
dataset.compareYyyy,
dataset.targetYyyy,
dataset.memo,
new CaseBuilder()
.when(datasetObjEntity.targetClassCd.eq("building"))
.then(1)
.otherwise(0)
.sum(),
new CaseBuilder()
.when(datasetObjEntity.targetClassCd.eq("container"))
.then(1)
.otherwise(0)
.sum(),
// ===== before (join으로) =====
beforeMaster.modelNo,
beforeDataset.id,
beforeDataset.uuid,
beforeDataset.dataType,
beforeDataset.title,
beforeDataset.roundNo,
beforeDataset.compareYyyy,
beforeDataset.targetYyyy,
beforeDataset.memo,
new CaseBuilder()
.when(beforeObj.targetClassCd.eq("building"))
.then(1)
.otherwise(0)
.sum(),
new CaseBuilder()
.when(beforeObj.targetClassCd.eq("container"))
.then(1)
.otherwise(0)
.sum()))
.from(modelMasterEntity)
// ===== 현재 dataset join =====
.leftJoin(modelDatasetMappEntity)
.on(modelDatasetMappEntity.modelUid.eq(modelMasterEntity.id))
.leftJoin(dataset)
.on(modelDatasetMappEntity.datasetUid.eq(dataset.id))
.leftJoin(datasetObjEntity)
.on(dataset.id.eq(datasetObjEntity.datasetUid))
// ===== before 모델 join =====
.leftJoin(beforeMaster)
.on(beforeMaster.id.eq(modelMasterEntity.beforeModelId))
.leftJoin(beforeMapp)
.on(beforeMapp.modelUid.eq(beforeMaster.id))
.leftJoin(beforeDataset)
.on(beforeMapp.datasetUid.eq(beforeDataset.id))
.leftJoin(beforeObj)
.on(beforeDataset.id.eq(beforeObj.datasetUid))
.where(modelMasterEntity.id.eq(modelId))
.groupBy(
modelMasterEntity.modelNo,
dataset.id,
dataset.uuid,
dataset.dataType,
dataset.title,
dataset.roundNo,
dataset.compareYyyy,
dataset.targetYyyy,
dataset.memo,
beforeMaster.modelNo,
beforeDataset.id,
beforeDataset.uuid,
beforeDataset.dataType,
beforeDataset.title,
beforeDataset.roundNo,
beforeDataset.compareYyyy,
beforeDataset.targetYyyy,
beforeDataset.memo)
.orderBy(dataset.createdDttm.desc())
.fetch();
}
@Override @Override
public List<SelectDataSet> getDatasetSelectG2G3List(DatasetReq req) { public List<SelectDataSet> getDatasetSelectG2G3List(DatasetReq req) {
BooleanBuilder builder = new BooleanBuilder(); BooleanBuilder builder = new BooleanBuilder();
builder.and(dataset.deleted.isFalse());
NumberExpression<Long> selectedCnt = null; NumberExpression<Long> selectedCnt = null;
NumberExpression<Long> wasteCnt = NumberExpression<Long> wasteCnt =
@@ -174,6 +286,7 @@ public class DatasetRepositoryImpl implements DatasetRepositoryCustom {
.select( .select(
Projections.constructor( Projections.constructor(
SelectDataSet.class, SelectDataSet.class,
Expressions.constant(req.getModelNo()),
dataset.id, dataset.id,
dataset.uuid, dataset.uuid,
dataset.dataType, dataset.dataType,
@@ -198,6 +311,116 @@ public class DatasetRepositoryImpl implements DatasetRepositoryCustom {
.fetch(); .fetch();
} }
@Override
public List<SelectTransferDataSet> getDatasetTransferSelectG2G3List(
Long modelId, String modelNo) {
// before join용
QModelMasterEntity beforeMaster = new QModelMasterEntity("beforeMaster");
QModelDatasetMappEntity beforeMapp = new QModelDatasetMappEntity("beforeMapp");
QDatasetEntity beforeDataset = new QDatasetEntity("beforeDataset");
QDatasetObjEntity beforeObj = new QDatasetObjEntity("beforeObj");
BooleanBuilder builder = new BooleanBuilder();
NumberExpression<Long> wasteCnt =
datasetObjEntity.targetClassCd.when("waste").then(1L).otherwise(0L).sum();
NumberExpression<Long> elseCnt =
new CaseBuilder()
.when(datasetObjEntity.targetClassCd.notIn("building", "container", "waste"))
.then(1L)
.otherwise(0L)
.sum();
NumberExpression<Long> selectedCnt = ModelType.G2.getId().equals(modelNo) ? wasteCnt : elseCnt;
// before도 동일 로직으로 cnt 계산
NumberExpression<Long> beforeWasteCnt =
beforeObj.targetClassCd.when("waste").then(1L).otherwise(0L).sum();
NumberExpression<Long> beforeElseCnt =
new CaseBuilder()
.when(beforeObj.targetClassCd.notIn("building", "container", "waste"))
.then(1L)
.otherwise(0L)
.sum();
NumberExpression<Long> beforeSelectedCnt =
ModelType.G2.getId().equals(modelNo) ? beforeWasteCnt : beforeElseCnt;
return queryFactory
.select(
Projections.constructor(
SelectTransferDataSet.class,
// ===== 현재 =====
modelMasterEntity.modelNo, // modelNo 파라미터 사용 (req.getModelNo() 제거)
dataset.id,
dataset.uuid,
dataset.dataType,
dataset.title,
dataset.roundNo,
dataset.compareYyyy,
dataset.targetYyyy,
dataset.memo,
selectedCnt, // classCount 자리에 들어가는 cnt (Long)
// ===== before =====
beforeMaster.modelNo,
beforeDataset.id,
beforeDataset.uuid,
beforeDataset.dataType,
beforeDataset.title,
beforeDataset.roundNo,
beforeDataset.compareYyyy,
beforeDataset.targetYyyy,
beforeDataset.memo,
beforeSelectedCnt))
.from(modelMasterEntity)
// ===== 현재 dataset =====
.leftJoin(modelDatasetMappEntity)
.on(modelDatasetMappEntity.modelUid.eq(modelMasterEntity.id))
.leftJoin(dataset)
.on(modelDatasetMappEntity.datasetUid.eq(dataset.id))
.leftJoin(datasetObjEntity)
.on(dataset.id.eq(datasetObjEntity.datasetUid))
// ===== before dataset =====
.leftJoin(beforeMaster)
.on(beforeMaster.id.eq(modelMasterEntity.beforeModelId))
.leftJoin(beforeMapp)
.on(beforeMapp.modelUid.eq(beforeMaster.id))
.leftJoin(beforeDataset)
.on(beforeMapp.datasetUid.eq(beforeDataset.id))
.leftJoin(beforeObj)
.on(beforeDataset.id.eq(beforeObj.datasetUid))
.where(modelMasterEntity.id.eq(modelId).and(builder))
// sum() 때문에 groupBy 필요
.groupBy(
dataset.id,
dataset.uuid,
dataset.dataType,
dataset.title,
dataset.roundNo,
dataset.compareYyyy,
dataset.targetYyyy,
dataset.memo,
beforeMaster.modelNo,
beforeDataset.id,
beforeDataset.uuid,
beforeDataset.dataType,
beforeDataset.title,
beforeDataset.roundNo,
beforeDataset.compareYyyy,
beforeDataset.targetYyyy,
beforeDataset.memo)
.orderBy(dataset.createdDttm.desc())
.fetch();
}
@Override @Override
public Long getDatasetMaxStage(int compareYyyy, int targetYyyy) { public Long getDatasetMaxStage(int compareYyyy, int targetYyyy) {
return queryFactory return queryFactory
@@ -236,7 +459,21 @@ public class DatasetRepositoryImpl implements DatasetRepositoryCustom {
return queryFactory return queryFactory
.select(dataset.id) .select(dataset.id)
.from(dataset) .from(dataset)
.where(dataset.uid.eq(mngRegDto.getUid())) .where(dataset.uid.eq(mngRegDto.getUid()), dataset.deleted.isFalse())
.fetchOne();
}
@Override
public List<String> findDatasetUid(List<Long> datasetIds) {
return queryFactory.select(dataset.uid).from(dataset).where(dataset.id.in(datasetIds)).fetch();
}
@Override
public Long findDatasetByUidExistsCnt(String uid) {
return queryFactory
.select(dataset.id.count())
.from(dataset)
.where(dataset.uid.eq(uid), dataset.deleted.isFalse())
.fetchOne(); .fetchOne();
} }
} }

View File

@@ -1,7 +1,10 @@
package com.kamco.cd.training.postgres.repository.hyperparam; package com.kamco.cd.training.postgres.repository.hyperparam;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto; import com.kamco.cd.training.hyperparam.dto.HyperParamDto;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto.SearchReq;
import com.kamco.cd.training.postgres.entity.ModelHyperParamEntity; import com.kamco.cd.training.postgres.entity.ModelHyperParamEntity;
import java.util.List;
import java.util.Optional; import java.util.Optional;
import java.util.UUID; import java.util.UUID;
import org.springframework.data.domain.Page; import org.springframework.data.domain.Page;
@@ -13,11 +16,41 @@ public interface HyperParamRepositoryCustom {
* *
* @return * @return
*/ */
@Deprecated
Optional<ModelHyperParamEntity> findHyperParamVer(); Optional<ModelHyperParamEntity> findHyperParamVer();
/**
* 모델 타입별 마지막 버전 조회
*
* @param modelType 모델 타입
* @return
*/
Optional<ModelHyperParamEntity> findHyperParamVerByModelType(ModelType modelType);
Optional<ModelHyperParamEntity> findHyperParamByHyperVer(String hyperVer); Optional<ModelHyperParamEntity> findHyperParamByHyperVer(String hyperVer);
/**
* 하이퍼 파라미터 상세조회
*
* @param uuid
* @return
*/
Optional<ModelHyperParamEntity> findHyperParamByUuid(UUID uuid); Optional<ModelHyperParamEntity> findHyperParamByUuid(UUID uuid);
Page<HyperParamDto.List> findByHyperVerList(HyperParamDto.SearchReq req); /**
* 하이퍼 파라미터 목록 조회
*
* @param model
* @param req
* @return
*/
Page<HyperParamDto.List> findByHyperVerList(ModelType model, SearchReq req);
/**
* 하이퍼 파라미터 모델타입으로 조회
*
* @param modelType
* @return
*/
List<ModelHyperParamEntity> getHyperParamByType(ModelType modelType);
} }

View File

@@ -2,12 +2,13 @@ package com.kamco.cd.training.postgres.repository.hyperparam;
import static com.kamco.cd.training.postgres.entity.QModelHyperParamEntity.modelHyperParamEntity; import static com.kamco.cd.training.postgres.entity.QModelHyperParamEntity.modelHyperParamEntity;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto; import com.kamco.cd.training.hyperparam.dto.HyperParamDto;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto.HyperType; import com.kamco.cd.training.hyperparam.dto.HyperParamDto.HyperType;
import com.kamco.cd.training.hyperparam.dto.HyperParamDto.SearchReq;
import com.kamco.cd.training.postgres.entity.ModelHyperParamEntity; import com.kamco.cd.training.postgres.entity.ModelHyperParamEntity;
import com.querydsl.core.BooleanBuilder; import com.querydsl.core.BooleanBuilder;
import com.querydsl.core.types.Projections; import com.querydsl.core.types.Projections;
import com.querydsl.core.types.dsl.NumberExpression;
import com.querydsl.jpa.impl.JPAQuery; import com.querydsl.jpa.impl.JPAQuery;
import com.querydsl.jpa.impl.JPAQueryFactory; import com.querydsl.jpa.impl.JPAQueryFactory;
import java.time.ZoneId; import java.time.ZoneId;
@@ -41,6 +42,23 @@ public class HyperParamRepositoryImpl implements HyperParamRepositoryCustom {
.fetchOne()); .fetchOne());
} }
@Override
public Optional<ModelHyperParamEntity> findHyperParamVerByModelType(ModelType modelType) {
return Optional.ofNullable(
queryFactory
.select(modelHyperParamEntity)
.from(modelHyperParamEntity)
.where(
modelHyperParamEntity
.delYn
.isFalse()
.and(modelHyperParamEntity.modelType.eq(modelType)))
.orderBy(modelHyperParamEntity.hyperVer.desc())
.limit(1)
.fetchOne());
}
@Override @Override
public Optional<ModelHyperParamEntity> findHyperParamByHyperVer(String hyperVer) { public Optional<ModelHyperParamEntity> findHyperParamByHyperVer(String hyperVer) {
@@ -63,17 +81,22 @@ public class HyperParamRepositoryImpl implements HyperParamRepositoryCustom {
queryFactory queryFactory
.select(modelHyperParamEntity) .select(modelHyperParamEntity)
.from(modelHyperParamEntity) .from(modelHyperParamEntity)
.where(modelHyperParamEntity.delYn.isFalse().and(modelHyperParamEntity.uuid.eq(uuid))) .where(modelHyperParamEntity.uuid.eq(uuid))
.fetchOne()); .fetchOne());
} }
@Override @Override
public Page<HyperParamDto.List> findByHyperVerList(HyperParamDto.SearchReq req) { public Page<HyperParamDto.List> findByHyperVerList(ModelType model, SearchReq req) {
Pageable pageable = req.toPageable(); Pageable pageable = req.toPageable();
BooleanBuilder builder = new BooleanBuilder(); BooleanBuilder builder = new BooleanBuilder();
builder.and(modelHyperParamEntity.delYn.isFalse()); builder.and(modelHyperParamEntity.delYn.isFalse());
if (model != null) {
builder.and(modelHyperParamEntity.modelType.eq(model));
}
if (req.getHyperVer() != null && !req.getHyperVer().isEmpty()) { if (req.getHyperVer() != null && !req.getHyperVer().isEmpty()) {
// 버전 // 버전
builder.and(modelHyperParamEntity.hyperVer.contains(req.getHyperVer())); builder.and(modelHyperParamEntity.hyperVer.contains(req.getHyperVer()));
@@ -96,26 +119,18 @@ public class HyperParamRepositoryImpl implements HyperParamRepositoryCustom {
} }
} }
NumberExpression<Long> totalUseCnt =
modelHyperParamEntity
.m1UseCnt
.coalesce(0L)
.add(modelHyperParamEntity.m2UseCnt.coalesce(0L))
.add(modelHyperParamEntity.m3UseCnt.coalesce(0L));
JPAQuery<HyperParamDto.List> query = JPAQuery<HyperParamDto.List> query =
queryFactory queryFactory
.select( .select(
Projections.constructor( Projections.constructor(
HyperParamDto.List.class, HyperParamDto.List.class,
modelHyperParamEntity.uuid, modelHyperParamEntity.uuid,
modelHyperParamEntity.modelType.as("model"),
modelHyperParamEntity.hyperVer, modelHyperParamEntity.hyperVer,
modelHyperParamEntity.createdDttm, modelHyperParamEntity.createdDttm,
modelHyperParamEntity.lastUsedDttm, modelHyperParamEntity.lastUsedDttm,
modelHyperParamEntity.m1UseCnt, modelHyperParamEntity.memo,
modelHyperParamEntity.m2UseCnt, modelHyperParamEntity.totalUseCnt))
modelHyperParamEntity.m3UseCnt,
totalUseCnt.as("totalUseCnt")))
.from(modelHyperParamEntity) .from(modelHyperParamEntity)
.where(builder); .where(builder);
@@ -140,8 +155,11 @@ public class HyperParamRepositoryImpl implements HyperParamRepositoryCustom {
asc asc
? modelHyperParamEntity.lastUsedDttm.asc() ? modelHyperParamEntity.lastUsedDttm.asc()
: modelHyperParamEntity.lastUsedDttm.desc()); : modelHyperParamEntity.lastUsedDttm.desc());
case "totalUseCnt" ->
case "totalUseCnt" -> query.orderBy(asc ? totalUseCnt.asc() : totalUseCnt.desc()); query.orderBy(
asc
? modelHyperParamEntity.totalUseCnt.asc()
: modelHyperParamEntity.totalUseCnt.desc());
default -> query.orderBy(modelHyperParamEntity.createdDttm.desc()); default -> query.orderBy(modelHyperParamEntity.createdDttm.desc());
} }
@@ -161,4 +179,17 @@ public class HyperParamRepositoryImpl implements HyperParamRepositoryCustom {
return new PageImpl<>(content, pageable, totalCount); return new PageImpl<>(content, pageable, totalCount);
} }
@Override
public List<ModelHyperParamEntity> getHyperParamByType(ModelType modelType) {
return queryFactory
.select(modelHyperParamEntity)
.from(modelHyperParamEntity)
.where(
modelHyperParamEntity
.delYn
.isFalse()
.and(modelHyperParamEntity.modelType.eq(modelType)))
.fetch();
}
} }

View File

@@ -1,6 +1,7 @@
package com.kamco.cd.training.postgres.repository.log; package com.kamco.cd.training.postgres.repository.log;
import com.kamco.cd.training.log.dto.AuditLogDto; import com.kamco.cd.training.log.dto.AuditLogDto;
import com.kamco.cd.training.log.dto.AuditLogDto.DownloadReq;
import java.time.LocalDate; import java.time.LocalDate;
import org.springframework.data.domain.Page; import org.springframework.data.domain.Page;
@@ -15,6 +16,9 @@ public interface AuditLogRepositoryCustom {
Page<AuditLogDto.UserAuditList> findLogByAccount( Page<AuditLogDto.UserAuditList> findLogByAccount(
AuditLogDto.searchReq searchReq, String searchValue); AuditLogDto.searchReq searchReq, String searchValue);
Page<AuditLogDto.DownloadRes> findDownloadLog(
AuditLogDto.searchReq searchReq, DownloadReq downloadReq);
Page<AuditLogDto.DailyDetail> findLogByDailyResult( Page<AuditLogDto.DailyDetail> findLogByDailyResult(
AuditLogDto.searchReq searchReq, LocalDate logDate); AuditLogDto.searchReq searchReq, LocalDate logDate);

View File

@@ -6,32 +6,42 @@ import static com.kamco.cd.training.postgres.entity.QMemberEntity.memberEntity;
import static com.kamco.cd.training.postgres.entity.QMenuEntity.menuEntity; import static com.kamco.cd.training.postgres.entity.QMenuEntity.menuEntity;
import com.kamco.cd.training.log.dto.AuditLogDto; import com.kamco.cd.training.log.dto.AuditLogDto;
import com.kamco.cd.training.log.dto.AuditLogDto.DownloadReq;
import com.kamco.cd.training.log.dto.AuditLogDto.searchReq;
import com.kamco.cd.training.log.dto.ErrorLogDto; import com.kamco.cd.training.log.dto.ErrorLogDto;
import com.kamco.cd.training.log.dto.EventStatus; import com.kamco.cd.training.log.dto.EventStatus;
import com.kamco.cd.training.log.dto.EventType; import com.kamco.cd.training.log.dto.EventType;
import com.kamco.cd.training.postgres.entity.AuditLogEntity;
import com.kamco.cd.training.postgres.entity.QMenuEntity; import com.kamco.cd.training.postgres.entity.QMenuEntity;
import com.querydsl.core.BooleanBuilder;
import com.querydsl.core.types.Projections; import com.querydsl.core.types.Projections;
import com.querydsl.core.types.dsl.*; import com.querydsl.core.types.dsl.*;
import com.querydsl.jpa.impl.JPAQueryFactory; import com.querydsl.jpa.impl.JPAQueryFactory;
import io.micrometer.common.util.StringUtils; import io.micrometer.common.util.StringUtils;
import java.time.LocalDate; import java.time.LocalDate;
import java.time.LocalDateTime; import java.time.ZoneId;
import java.time.ZonedDateTime; import java.time.ZonedDateTime;
import java.util.List; import java.util.List;
import java.util.Objects; import java.util.Objects;
import lombok.RequiredArgsConstructor;
import org.springframework.data.domain.Page; import org.springframework.data.domain.Page;
import org.springframework.data.domain.PageImpl; import org.springframework.data.domain.PageImpl;
import org.springframework.data.domain.Pageable; import org.springframework.data.domain.Pageable;
import org.springframework.data.jpa.repository.support.QuerydslRepositorySupport;
import org.springframework.stereotype.Repository; import org.springframework.stereotype.Repository;
@Repository @Repository
@RequiredArgsConstructor public class AuditLogRepositoryImpl extends QuerydslRepositorySupport
public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom { implements AuditLogRepositoryCustom {
private static final ZoneId ZONE = ZoneId.of("Asia/Seoul");
private final JPAQueryFactory queryFactory; private final JPAQueryFactory queryFactory;
private final StringExpression NULL_STRING = Expressions.stringTemplate("cast(null as text)"); private final StringExpression NULL_STRING = Expressions.stringTemplate("cast(null as text)");
public AuditLogRepositoryImpl(JPAQueryFactory queryFactory) {
super(AuditLogEntity.class);
this.queryFactory = queryFactory;
}
@Override @Override
public Page<AuditLogDto.DailyAuditList> findLogByDaily( public Page<AuditLogDto.DailyAuditList> findLogByDaily(
AuditLogDto.searchReq searchReq, LocalDate startDate, LocalDate endDate) { AuditLogDto.searchReq searchReq, LocalDate startDate, LocalDate endDate) {
@@ -87,7 +97,7 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
.from(auditLogEntity) .from(auditLogEntity)
.leftJoin(menuEntity) .leftJoin(menuEntity)
.on(auditLogEntity.menuUid.eq(menuEntity.menuUid)) .on(auditLogEntity.menuUid.eq(menuEntity.menuUid))
.where(menuNameEquals(searchValue)) .where(auditLogEntity.menuUid.ne("SYSTEM"), menuNameEquals(searchValue))
.groupBy(auditLogEntity.menuUid) .groupBy(auditLogEntity.menuUid)
.offset(pageable.getOffset()) .offset(pageable.getOffset())
.limit(pageable.getPageSize()) .limit(pageable.getPageSize())
@@ -128,7 +138,7 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
.from(auditLogEntity) .from(auditLogEntity)
.leftJoin(memberEntity) .leftJoin(memberEntity)
.on(auditLogEntity.userUid.eq(memberEntity.id)) .on(auditLogEntity.userUid.eq(memberEntity.id))
.where(loginIdOrUsernameContains(searchValue)) .where(auditLogEntity.userUid.isNotNull(), loginIdOrUsernameContains(searchValue))
.groupBy(auditLogEntity.userUid, memberEntity.employeeNo, memberEntity.name) .groupBy(auditLogEntity.userUid, memberEntity.employeeNo, memberEntity.name)
.offset(pageable.getOffset()) .offset(pageable.getOffset())
.limit(pageable.getPageSize()) .limit(pageable.getPageSize())
@@ -147,6 +157,62 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
return new PageImpl<>(foundContent, pageable, countQuery); return new PageImpl<>(foundContent, pageable, countQuery);
} }
@Override
public Page<AuditLogDto.DownloadRes> findDownloadLog(
AuditLogDto.searchReq searchReq, DownloadReq req) {
Pageable pageable = searchReq.toPageable();
BooleanBuilder whereBuilder = new BooleanBuilder();
whereBuilder.and(auditLogEntity.eventStatus.ne(EventStatus.valueOf("FAILED")));
whereBuilder.and(auditLogEntity.eventType.eq(EventType.valueOf("DOWNLOAD")));
// if (req.getMenuId() != null && !req.getMenuId().isEmpty()) {
// whereBuilder.and(auditLogEntity.menuUid.eq(req.getMenuId()));
// }
if (req.getUuid() != null) {
whereBuilder.and(auditLogEntity.requestUri.contains(req.getRequestUri()));
whereBuilder.and(auditLogEntity.downloadUuid.eq(req.getUuid()));
}
if (req.getSearchValue() != null && !req.getSearchValue().isEmpty()) {
whereBuilder.and(
memberEntity
.name
.contains(req.getSearchValue())
.or(memberEntity.employeeNo.contains(req.getSearchValue())));
}
List<AuditLogDto.DownloadRes> foundContent =
queryFactory
.select(
Projections.constructor(
AuditLogDto.DownloadRes.class,
memberEntity.name,
memberEntity.employeeNo,
auditLogEntity.createdDate.as("downloadDttm")))
.from(auditLogEntity)
.leftJoin(memberEntity)
.on(auditLogEntity.userUid.eq(memberEntity.id))
.where(whereBuilder, createdDateBetween(req.getStartDate(), req.getEndDate()))
.offset(pageable.getOffset())
.limit(pageable.getPageSize())
.orderBy(auditLogEntity.createdDate.desc())
.fetch();
Long countQuery =
queryFactory
.select(auditLogEntity.userUid.countDistinct())
.from(auditLogEntity)
.leftJoin(memberEntity)
.on(auditLogEntity.userUid.eq(memberEntity.id))
.where(whereBuilder, createdDateBetween(req.getStartDate(), req.getEndDate()))
.fetchOne();
return new PageImpl<>(foundContent, pageable, countQuery);
}
@Override @Override
public Page<AuditLogDto.DailyDetail> findLogByDailyResult( public Page<AuditLogDto.DailyDetail> findLogByDailyResult(
AuditLogDto.searchReq searchReq, LocalDate logDate) { AuditLogDto.searchReq searchReq, LocalDate logDate) {
@@ -176,6 +242,9 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
memberEntity.employeeNo.as("loginId"), memberEntity.employeeNo.as("loginId"),
menuEntity.menuNm.as("menuName"), menuEntity.menuNm.as("menuName"),
auditLogEntity.eventType.as("eventType"), auditLogEntity.eventType.as("eventType"),
Expressions.stringTemplate(
"to_char({0}, 'YYYY-MM-DD HH24:MI')", auditLogEntity.createdDate)
.as("logDateTime"),
Projections.constructor( Projections.constructor(
AuditLogDto.LogDetail.class, AuditLogDto.LogDetail.class,
Expressions.constant("한국자산관리공사"), // serviceName Expressions.constant("한국자산관리공사"), // serviceName
@@ -184,7 +253,7 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
menuEntity.menuUrl.as("menuUrl"), menuEntity.menuUrl.as("menuUrl"),
menuEntity.description.as("menuDescription"), menuEntity.description.as("menuDescription"),
menuEntity.menuOrder.as("sortOrder"), menuEntity.menuOrder.as("sortOrder"),
menuEntity.isUse.as("used")))) menuEntity.isUse.as("used")))) // TODO
.from(auditLogEntity) .from(auditLogEntity)
.leftJoin(menuEntity) .leftJoin(menuEntity)
.on(auditLogEntity.menuUid.eq(menuEntity.menuUid)) .on(auditLogEntity.menuUid.eq(menuEntity.menuUid))
@@ -238,8 +307,8 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
AuditLogDto.MenuDetail.class, AuditLogDto.MenuDetail.class,
auditLogEntity.id.as("logId"), auditLogEntity.id.as("logId"),
Expressions.stringTemplate( Expressions.stringTemplate(
"to_char({0}, 'YYYY-MM-DD')", auditLogEntity.createdDate) "to_char({0}, 'YYYY-MM-DD HH24:MI')", auditLogEntity.createdDate)
.as("logDateTime"), // ?? .as("logDateTime"),
memberEntity.name.as("userName"), memberEntity.name.as("userName"),
memberEntity.employeeNo.as("loginId"), memberEntity.employeeNo.as("loginId"),
auditLogEntity.eventType.as("eventType"), auditLogEntity.eventType.as("eventType"),
@@ -305,7 +374,7 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
AuditLogDto.UserDetail.class, AuditLogDto.UserDetail.class,
auditLogEntity.id.as("logId"), auditLogEntity.id.as("logId"),
Expressions.stringTemplate( Expressions.stringTemplate(
"to_char({0}, 'YYYY-MM-DD')", auditLogEntity.createdDate) "to_char({0}, 'YYYY-MM-DD HH24:MI')", auditLogEntity.createdDate)
.as("logDateTime"), .as("logDateTime"),
menuEntity.menuNm.as("menuName"), menuEntity.menuNm.as("menuName"),
auditLogEntity.eventType.as("eventType"), auditLogEntity.eventType.as("eventType"),
@@ -349,12 +418,23 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
if (Objects.isNull(startDate) || Objects.isNull(endDate)) { if (Objects.isNull(startDate) || Objects.isNull(endDate)) {
return null; return null;
} }
LocalDateTime startDateTime = startDate.atStartOfDay(); ZoneId zoneId = ZoneId.of("Asia/Seoul");
LocalDateTime endDateTime = endDate.plusDays(1).atStartOfDay(); ZonedDateTime startDateTime = startDate.atStartOfDay(zoneId);
ZonedDateTime endDateTime = endDate.plusDays(1).atStartOfDay(zoneId);
return auditLogEntity return auditLogEntity
.createdDate .createdDate
.goe(ZonedDateTime.from(startDateTime)) .goe(startDateTime)
.and(auditLogEntity.createdDate.lt(ZonedDateTime.from(endDateTime))); .and(auditLogEntity.createdDate.lt(endDateTime));
}
private BooleanExpression createdDateBetween(LocalDate startDate, LocalDate endDate) {
if (startDate == null || endDate == null) {
return null;
}
ZonedDateTime start = startDate.atStartOfDay(ZONE);
ZonedDateTime endExclusive = endDate.plusDays(1).atStartOfDay(ZONE);
return auditLogEntity.createdDate.goe(start).and(auditLogEntity.createdDate.lt(endExclusive));
} }
private BooleanExpression menuNameEquals(String searchValue) { private BooleanExpression menuNameEquals(String searchValue) {
@@ -393,11 +473,11 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
} }
private BooleanExpression eventEndedAtEqDate(LocalDate logDate) { private BooleanExpression eventEndedAtEqDate(LocalDate logDate) {
StringExpression eventEndedDate = ZoneId zoneId = ZoneId.of("Asia/Seoul");
Expressions.stringTemplate("to_char({0}, 'YYYY-MM-DD')", auditLogEntity.createdDate); ZonedDateTime start = logDate.atStartOfDay(zoneId);
LocalDateTime comparisonDate = logDate.atStartOfDay(); ZonedDateTime end = logDate.plusDays(1).atStartOfDay(zoneId);
return eventEndedDate.eq(comparisonDate.toString()); return auditLogEntity.createdDate.goe(start).and(auditLogEntity.createdDate.lt(end));
} }
private BooleanExpression menuUidEq(String menuUid) { private BooleanExpression menuUidEq(String menuUid) {
@@ -410,7 +490,7 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
private NumberExpression<Integer> readCount() { private NumberExpression<Integer> readCount() {
return new CaseBuilder() return new CaseBuilder()
.when(auditLogEntity.eventType.eq(EventType.READ)) .when(auditLogEntity.eventType.in(EventType.LIST, EventType.DETAIL))
.then(1) .then(1)
.otherwise(0) .otherwise(0)
.sum(); .sum();
@@ -418,7 +498,7 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
private NumberExpression<Integer> cudCount() { private NumberExpression<Integer> cudCount() {
return new CaseBuilder() return new CaseBuilder()
.when(auditLogEntity.eventType.in(EventType.CREATE, EventType.UPDATE, EventType.DELETE)) .when(auditLogEntity.eventType.in(EventType.ADDED, EventType.MODIFIED, EventType.REMOVE))
.then(1) .then(1)
.otherwise(0) .otherwise(0)
.sum(); .sum();
@@ -426,7 +506,7 @@ public class AuditLogRepositoryImpl implements AuditLogRepositoryCustom {
private NumberExpression<Integer> printCount() { private NumberExpression<Integer> printCount() {
return new CaseBuilder() return new CaseBuilder()
.when(auditLogEntity.eventType.eq(EventType.PRINT)) .when(auditLogEntity.eventType.eq(EventType.OTHER))
.then(1) .then(1)
.otherwise(0) .otherwise(0)
.sum(); .sum();

View File

@@ -8,29 +8,35 @@ import static com.kamco.cd.training.postgres.entity.QMenuEntity.menuEntity;
import com.kamco.cd.training.log.dto.ErrorLogDto; import com.kamco.cd.training.log.dto.ErrorLogDto;
import com.kamco.cd.training.log.dto.EventStatus; import com.kamco.cd.training.log.dto.EventStatus;
import com.kamco.cd.training.log.dto.EventType; import com.kamco.cd.training.log.dto.EventType;
import com.kamco.cd.training.postgres.entity.AuditLogEntity;
import com.querydsl.core.types.Projections; import com.querydsl.core.types.Projections;
import com.querydsl.core.types.dsl.BooleanExpression; import com.querydsl.core.types.dsl.BooleanExpression;
import com.querydsl.core.types.dsl.Expressions; import com.querydsl.core.types.dsl.Expressions;
import com.querydsl.core.types.dsl.StringExpression; import com.querydsl.core.types.dsl.StringExpression;
import com.querydsl.jpa.impl.JPAQueryFactory; import com.querydsl.jpa.impl.JPAQueryFactory;
import java.time.LocalDate; import java.time.LocalDate;
import java.time.LocalDateTime; import java.time.ZoneId;
import java.time.ZonedDateTime; import java.time.ZonedDateTime;
import java.util.List; import java.util.List;
import java.util.Objects; import java.util.Objects;
import lombok.RequiredArgsConstructor;
import org.springframework.data.domain.Page; import org.springframework.data.domain.Page;
import org.springframework.data.domain.PageImpl; import org.springframework.data.domain.PageImpl;
import org.springframework.data.domain.Pageable; import org.springframework.data.domain.Pageable;
import org.springframework.data.jpa.repository.support.QuerydslRepositorySupport;
import org.springframework.stereotype.Repository; import org.springframework.stereotype.Repository;
@Repository @Repository
@RequiredArgsConstructor public class ErrorLogRepositoryImpl extends QuerydslRepositorySupport
public class ErrorLogRepositoryImpl implements ErrorLogRepositoryCustom { implements ErrorLogRepositoryCustom {
private final JPAQueryFactory queryFactory; private final JPAQueryFactory queryFactory;
private final StringExpression NULL_STRING = Expressions.stringTemplate("cast(null as text)"); private final StringExpression NULL_STRING = Expressions.stringTemplate("cast(null as text)");
public ErrorLogRepositoryImpl(JPAQueryFactory queryFactory) {
super(AuditLogEntity.class);
this.queryFactory = queryFactory;
}
@Override @Override
public Page<ErrorLogDto.Basic> findLogByError(ErrorLogDto.ErrorSearchReq searchReq) { public Page<ErrorLogDto.Basic> findLogByError(ErrorLogDto.ErrorSearchReq searchReq) {
Pageable pageable = searchReq.toPageable(); Pageable pageable = searchReq.toPageable();
@@ -52,7 +58,7 @@ public class ErrorLogRepositoryImpl implements ErrorLogRepositoryCustom {
errorLogEntity.errorMessage.as("errorMessage"), errorLogEntity.errorMessage.as("errorMessage"),
errorLogEntity.stackTrace.as("errorDetail"), errorLogEntity.stackTrace.as("errorDetail"),
Expressions.stringTemplate( Expressions.stringTemplate(
"to_char({0}, 'YYYY-MM-DD')", errorLogEntity.createdDate))) "to_char({0}, 'YYYY-MM-DD HH24:MI:SS.FF3')", errorLogEntity.createdDate)))
.from(errorLogEntity) .from(errorLogEntity)
.leftJoin(auditLogEntity) .leftJoin(auditLogEntity)
.on(errorLogEntity.id.eq(auditLogEntity.errorLogUid)) .on(errorLogEntity.id.eq(auditLogEntity.errorLogUid))
@@ -94,12 +100,14 @@ public class ErrorLogRepositoryImpl implements ErrorLogRepositoryCustom {
if (Objects.isNull(startDate) || Objects.isNull(endDate)) { if (Objects.isNull(startDate) || Objects.isNull(endDate)) {
return null; return null;
} }
LocalDateTime startDateTime = startDate.atStartOfDay();
LocalDateTime endDateTime = endDate.plusDays(1).atStartOfDay(); ZoneId zoneId = ZoneId.of("Asia/Seoul");
ZonedDateTime startDateTime = startDate.atStartOfDay(zoneId);
ZonedDateTime endDateTime = endDate.plusDays(1).atStartOfDay(zoneId);
return auditLogEntity return auditLogEntity
.createdDate .createdDate
.goe(ZonedDateTime.from(startDateTime)) .goe(startDateTime)
.and(auditLogEntity.createdDate.lt(ZonedDateTime.from(endDateTime))); .and(auditLogEntity.createdDate.lt(endDateTime));
} }
private BooleanExpression eventStatusEqFailed() { private BooleanExpression eventStatusEqFailed() {

View File

@@ -5,4 +5,6 @@ import java.util.Optional;
public interface ModelConfigRepositoryCustom { public interface ModelConfigRepositoryCustom {
Optional<ModelConfigDto.Basic> findModelConfigByModelId(Long modelId); Optional<ModelConfigDto.Basic> findModelConfigByModelId(Long modelId);
Optional<ModelConfigDto.TransferBasic> findModelTransferConfigByModelId(Long modelId);
} }

View File

@@ -1,8 +1,12 @@
package com.kamco.cd.training.postgres.repository.model; package com.kamco.cd.training.postgres.repository.model;
import static com.kamco.cd.training.postgres.entity.QModelConfigEntity.modelConfigEntity; import static com.kamco.cd.training.postgres.entity.QModelConfigEntity.modelConfigEntity;
import static com.kamco.cd.training.postgres.entity.QModelMasterEntity.modelMasterEntity;
import com.kamco.cd.training.model.dto.ModelConfigDto.Basic; import com.kamco.cd.training.model.dto.ModelConfigDto.Basic;
import com.kamco.cd.training.model.dto.ModelConfigDto.TransferBasic;
import com.kamco.cd.training.postgres.entity.QModelConfigEntity;
import com.kamco.cd.training.postgres.entity.QModelMasterEntity;
import com.querydsl.core.types.Projections; import com.querydsl.core.types.Projections;
import com.querydsl.jpa.impl.JPAQueryFactory; import com.querydsl.jpa.impl.JPAQueryFactory;
import java.util.Optional; import java.util.Optional;
@@ -34,4 +38,44 @@ public class ModelConfigRepositoryImpl implements ModelConfigRepositoryCustom {
.where(modelConfigEntity.model.id.eq(modelId)) .where(modelConfigEntity.model.id.eq(modelId))
.fetchOne()); .fetchOne());
} }
@Override
public Optional<TransferBasic> findModelTransferConfigByModelId(Long modelId) {
QModelConfigEntity beforeConfig = new QModelConfigEntity("beforeConfig");
QModelMasterEntity beforeMaster = new QModelMasterEntity("beforeMaster");
return Optional.ofNullable(
queryFactory
.select(
Projections.constructor(
TransferBasic.class,
// ===== 현재 =====
modelConfigEntity.id,
modelConfigEntity.model.id,
modelConfigEntity.epochCount,
modelConfigEntity.trainPercent,
modelConfigEntity.validationPercent,
modelConfigEntity.testPercent,
modelConfigEntity.memo,
// ===== before =====
beforeConfig.id,
beforeConfig.model.id,
beforeConfig.epochCount,
beforeConfig.trainPercent,
beforeConfig.validationPercent,
beforeConfig.testPercent,
beforeConfig.memo))
.from(modelConfigEntity)
.innerJoin(modelConfigEntity.model, modelMasterEntity)
// before 모델 조인
.leftJoin(beforeMaster)
.on(beforeMaster.id.eq(modelMasterEntity.beforeModelId))
.leftJoin(beforeConfig)
.on(beforeConfig.model.id.eq(beforeMaster.id))
.where(modelMasterEntity.id.eq(modelId))
.fetchOne());
}
} }

View File

@@ -6,4 +6,5 @@ import org.springframework.stereotype.Repository;
@Repository @Repository
public interface ModelDatasetMappRepository public interface ModelDatasetMappRepository
extends JpaRepository<ModelDatasetMappEntity, ModelDatasetMappEntity.ModelDatasetMappId> {} extends JpaRepository<ModelDatasetMappEntity, ModelDatasetMappEntity.ModelDatasetMappId>,
ModelDatasetMappRepositoryCustom {}

View File

@@ -0,0 +1,15 @@
package com.kamco.cd.training.postgres.repository.model;
import com.kamco.cd.training.postgres.entity.ModelDatasetMappEntity;
import com.kamco.cd.training.train.dto.ModelTrainLinkDto;
import java.util.List;
public interface ModelDatasetMappRepositoryCustom {
List<ModelDatasetMappEntity> findByModelUid(Long modelId);
List<ModelTrainLinkDto> findDatasetTrainPath(Long modelId);
List<ModelTrainLinkDto> findDatasetValPath(Long modelId);
List<ModelTrainLinkDto> findDatasetTestPath(Long modelId);
}

View File

@@ -0,0 +1,165 @@
package com.kamco.cd.training.postgres.repository.model;
import static com.kamco.cd.training.postgres.entity.QDatasetEntity.datasetEntity;
import static com.kamco.cd.training.postgres.entity.QModelDatasetMappEntity.modelDatasetMappEntity;
import static com.kamco.cd.training.postgres.entity.QModelMasterEntity.modelMasterEntity;
import com.kamco.cd.training.common.enums.ModelType;
import com.kamco.cd.training.postgres.entity.ModelDatasetMappEntity;
import com.kamco.cd.training.postgres.entity.QDatasetObjEntity;
import com.kamco.cd.training.postgres.entity.QDatasetTestObjEntity;
import com.kamco.cd.training.postgres.entity.QDatasetValObjEntity;
import com.kamco.cd.training.train.dto.ModelTrainLinkDto;
import com.querydsl.core.types.Projections;
import com.querydsl.jpa.impl.JPAQueryFactory;
import java.util.List;
import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Repository;
@Repository
@RequiredArgsConstructor
public class ModelDatasetMappRepositoryImpl implements ModelDatasetMappRepositoryCustom {
private final JPAQueryFactory queryFactory;
@Override
public List<ModelDatasetMappEntity> findByModelUid(Long modelId) {
return queryFactory
.select(modelDatasetMappEntity)
.from(modelDatasetMappEntity)
.where(modelDatasetMappEntity.modelUid.eq(modelId))
.fetch();
}
@Override
public List<ModelTrainLinkDto> findDatasetTrainPath(Long modelId) {
QDatasetObjEntity datasetObjEntity = QDatasetObjEntity.datasetObjEntity;
return queryFactory
.select(
Projections.constructor(
ModelTrainLinkDto.class,
modelMasterEntity.id,
modelMasterEntity.trainType,
modelMasterEntity.modelNo,
modelDatasetMappEntity.datasetUid,
datasetObjEntity.targetClassCd,
datasetObjEntity.comparePath,
datasetObjEntity.targetPath,
datasetObjEntity.labelPath,
datasetObjEntity.geojsonPath,
datasetEntity.uid))
.from(modelMasterEntity)
.leftJoin(modelDatasetMappEntity)
.on(modelDatasetMappEntity.modelUid.eq(modelMasterEntity.id))
.leftJoin(datasetEntity)
.on(datasetEntity.id.eq(modelDatasetMappEntity.datasetUid))
.leftJoin(datasetObjEntity)
.on(
datasetObjEntity
.datasetUid
.eq(modelDatasetMappEntity.datasetUid)
.and(
modelMasterEntity
.modelNo
.eq(ModelType.G1.getId())
.and(datasetObjEntity.targetClassCd.upper().in("CONTAINER", "BUILDING"))
.or(
modelMasterEntity
.modelNo
.eq(ModelType.G2.getId())
.and(datasetObjEntity.targetClassCd.upper().eq("WASTE")))
.or(modelMasterEntity.modelNo.eq(ModelType.G3.getId()))))
.where(modelMasterEntity.id.eq(modelId))
.fetch();
}
@Override
public List<ModelTrainLinkDto> findDatasetValPath(Long modelId) {
QDatasetValObjEntity datasetValObjEntity = QDatasetValObjEntity.datasetValObjEntity;
return queryFactory
.select(
Projections.constructor(
ModelTrainLinkDto.class,
modelMasterEntity.id,
modelMasterEntity.trainType,
modelMasterEntity.modelNo,
modelDatasetMappEntity.datasetUid,
datasetValObjEntity.targetClassCd,
datasetValObjEntity.comparePath,
datasetValObjEntity.targetPath,
datasetValObjEntity.labelPath,
datasetValObjEntity.geojsonPath,
datasetEntity.uid))
.from(modelMasterEntity)
.leftJoin(modelDatasetMappEntity)
.on(modelDatasetMappEntity.modelUid.eq(modelMasterEntity.id))
.leftJoin(datasetEntity)
.on(datasetEntity.id.eq(modelDatasetMappEntity.datasetUid))
.leftJoin(datasetValObjEntity)
.on(
datasetValObjEntity
.datasetUid
.eq(modelDatasetMappEntity.datasetUid)
.and(
modelMasterEntity
.modelNo
.eq(ModelType.G1.getId())
.and(datasetValObjEntity.targetClassCd.upper().in("CONTAINER", "BUILDING"))
.or(
modelMasterEntity
.modelNo
.eq(ModelType.G2.getId())
.and(datasetValObjEntity.targetClassCd.upper().eq("WASTE")))
.or(modelMasterEntity.modelNo.eq(ModelType.G3.getId()))))
.where(modelMasterEntity.id.eq(modelId))
.fetch();
}
@Override
public List<ModelTrainLinkDto> findDatasetTestPath(Long modelId) {
QDatasetTestObjEntity datasetTestObjEntity = QDatasetTestObjEntity.datasetTestObjEntity;
return queryFactory
.select(
Projections.constructor(
ModelTrainLinkDto.class,
modelMasterEntity.id,
modelMasterEntity.trainType,
modelMasterEntity.modelNo,
modelDatasetMappEntity.datasetUid,
datasetTestObjEntity.targetClassCd,
datasetTestObjEntity.comparePath,
datasetTestObjEntity.targetPath,
datasetTestObjEntity.labelPath,
datasetTestObjEntity.geojsonPath,
datasetEntity.uid))
.from(modelMasterEntity)
.leftJoin(modelDatasetMappEntity)
.on(modelDatasetMappEntity.modelUid.eq(modelMasterEntity.id))
.leftJoin(datasetEntity)
.on(datasetEntity.id.eq(modelDatasetMappEntity.datasetUid))
.leftJoin(datasetTestObjEntity)
.on(
datasetTestObjEntity
.datasetUid
.eq(modelDatasetMappEntity.datasetUid)
.and(
modelMasterEntity
.modelNo
.eq(ModelType.G1.getId())
.and(datasetTestObjEntity.targetClassCd.upper().in("CONTAINER", "BUILDING"))
.or(
modelMasterEntity
.modelNo
.eq(ModelType.G2.getId())
.and(datasetTestObjEntity.targetClassCd.upper().eq("WASTE")))
.or(modelMasterEntity.modelNo.eq(ModelType.G3.getId()))))
.where(modelMasterEntity.id.eq(modelId))
.fetch();
}
}

View File

@@ -3,6 +3,13 @@ package com.kamco.cd.training.postgres.repository.model;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.DetailSummary; import com.kamco.cd.training.model.dto.ModelTrainDetailDto.DetailSummary;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.HyperSummary; import com.kamco.cd.training.model.dto.ModelTrainDetailDto.HyperSummary;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.MappingDataset; import com.kamco.cd.training.model.dto.ModelTrainDetailDto.MappingDataset;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelBestEpoch;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelFileInfo;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelTestMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelTrainMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelValidationMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.TransferHyperSummary;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.ModelProgressStepDto;
import com.kamco.cd.training.postgres.entity.ModelMasterEntity; import com.kamco.cd.training.postgres.entity.ModelMasterEntity;
import java.util.List; import java.util.List;
import java.util.Optional; import java.util.Optional;
@@ -16,7 +23,23 @@ public interface ModelDetailRepositoryCustom {
HyperSummary getByModelHyperParamSummary(UUID uuid); HyperSummary getByModelHyperParamSummary(UUID uuid);
TransferHyperSummary getByModelTransferHyperParamSummary(UUID uuid);
List<MappingDataset> getByModelMappingDataset(UUID uuid); List<MappingDataset> getByModelMappingDataset(UUID uuid);
ModelMasterEntity findByModelByUUID(UUID uuid); ModelMasterEntity findByModelByUUID(UUID uuid);
List<ModelTrainMetrics> getModelTrainMetricResult(UUID uuid);
List<ModelValidationMetrics> getModelValidationMetricResult(UUID uuid);
List<ModelTestMetrics> getModelTestMetricResult(UUID uuid);
ModelBestEpoch getModelTrainBestEpoch(UUID uuid);
ModelFileInfo getModelTrainFileInfo(UUID uuid);
List<ModelProgressStepDto> findModelTrainProgressInfo(UUID uuid);
ModelMasterEntity findByModelBeforeId(Long beforeModelId);
} }

View File

@@ -5,20 +5,37 @@ import static com.kamco.cd.training.postgres.entity.QModelDatasetEntity.modelDat
import static com.kamco.cd.training.postgres.entity.QModelDatasetMappEntity.modelDatasetMappEntity; import static com.kamco.cd.training.postgres.entity.QModelDatasetMappEntity.modelDatasetMappEntity;
import static com.kamco.cd.training.postgres.entity.QModelHyperParamEntity.modelHyperParamEntity; import static com.kamco.cd.training.postgres.entity.QModelHyperParamEntity.modelHyperParamEntity;
import static com.kamco.cd.training.postgres.entity.QModelMasterEntity.modelMasterEntity; import static com.kamco.cd.training.postgres.entity.QModelMasterEntity.modelMasterEntity;
import static com.kamco.cd.training.postgres.entity.QModelMetricsTestEntity.modelMetricsTestEntity;
import static com.kamco.cd.training.postgres.entity.QModelMetricsTrainEntity.modelMetricsTrainEntity;
import static com.kamco.cd.training.postgres.entity.QModelMetricsValidationEntity.modelMetricsValidationEntity;
import com.kamco.cd.training.common.enums.TrainStatusType;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.DetailSummary; import com.kamco.cd.training.model.dto.ModelTrainDetailDto.DetailSummary;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.HyperSummary; import com.kamco.cd.training.model.dto.ModelTrainDetailDto.HyperSummary;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.MappingDataset; import com.kamco.cd.training.model.dto.ModelTrainDetailDto.MappingDataset;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelBestEpoch;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelFileInfo;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelTestMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelTrainMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.ModelValidationMetrics;
import com.kamco.cd.training.model.dto.ModelTrainDetailDto.TransferHyperSummary;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.ModelProgressStepDto;
import com.kamco.cd.training.postgres.entity.ModelMasterEntity; import com.kamco.cd.training.postgres.entity.ModelMasterEntity;
import com.kamco.cd.training.postgres.entity.QModelHyperParamEntity;
import com.kamco.cd.training.postgres.entity.QModelMasterEntity;
import com.querydsl.core.types.Expression;
import com.querydsl.core.types.Projections; import com.querydsl.core.types.Projections;
import com.querydsl.jpa.JPAExpressions; import com.querydsl.jpa.JPAExpressions;
import com.querydsl.jpa.impl.JPAQueryFactory; import com.querydsl.jpa.impl.JPAQueryFactory;
import java.util.ArrayList;
import java.util.List; import java.util.List;
import java.util.Optional; import java.util.Optional;
import java.util.UUID; import java.util.UUID;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Repository; import org.springframework.stereotype.Repository;
@Slf4j
@Repository @Repository
@RequiredArgsConstructor @RequiredArgsConstructor
public class ModelDetailRepositoryImpl implements ModelDetailRepositoryCustom { public class ModelDetailRepositoryImpl implements ModelDetailRepositoryCustom {
@@ -43,6 +60,13 @@ public class ModelDetailRepositoryImpl implements ModelDetailRepositoryCustom {
@Override @Override
public DetailSummary getModelDetailSummary(UUID uuid) { public DetailSummary getModelDetailSummary(UUID uuid) {
QModelMasterEntity beforeModel = new QModelMasterEntity("beforeModel"); // alias
Expression<UUID> beforeModelUuid =
com.querydsl.jpa.JPAExpressions.select(beforeModel.uuid)
.from(beforeModel)
.where(beforeModel.id.eq(modelMasterEntity.beforeModelId));
return queryFactory return queryFactory
.select( .select(
Projections.constructor( Projections.constructor(
@@ -54,7 +78,8 @@ public class ModelDetailRepositoryImpl implements ModelDetailRepositoryCustom {
modelMasterEntity.step1StrtDttm, modelMasterEntity.step1StrtDttm,
modelMasterEntity.step2EndDttm, modelMasterEntity.step2EndDttm,
modelMasterEntity.statusCd, modelMasterEntity.statusCd,
modelMasterEntity.trainType)) modelMasterEntity.trainType,
beforeModelUuid))
.from(modelMasterEntity) .from(modelMasterEntity)
.where(modelMasterEntity.uuid.eq(uuid)) .where(modelMasterEntity.uuid.eq(uuid))
.fetchOne(); .fetchOne();
@@ -82,6 +107,41 @@ public class ModelDetailRepositoryImpl implements ModelDetailRepositoryCustom {
.fetchOne(); .fetchOne();
} }
@Override
public TransferHyperSummary getByModelTransferHyperParamSummary(UUID uuid) {
QModelMasterEntity subMaster = new QModelMasterEntity("subMaster");
QModelHyperParamEntity subHyper = new QModelHyperParamEntity("subHyper");
return queryFactory
.select(
Projections.constructor(
TransferHyperSummary.class,
modelHyperParamEntity.uuid,
modelHyperParamEntity.id,
modelHyperParamEntity.hyperVer,
modelHyperParamEntity.backbone,
modelHyperParamEntity.inputSize,
modelHyperParamEntity.cropSize,
modelHyperParamEntity.batchSize,
subHyper.uuid,
subHyper.id,
subHyper.hyperVer,
subHyper.backbone,
subHyper.inputSize,
subHyper.cropSize,
subHyper.batchSize))
.from(modelMasterEntity)
.innerJoin(modelHyperParamEntity)
.on(modelHyperParamEntity.id.eq(modelMasterEntity.hyperParamId))
.leftJoin(subMaster)
.on(subMaster.id.eq(modelMasterEntity.beforeModelId))
.leftJoin(subHyper)
.on(subHyper.id.eq(subMaster.hyperParamId))
.where(modelMasterEntity.uuid.eq(uuid))
.fetchOne();
}
@Override @Override
public List<MappingDataset> getByModelMappingDataset(UUID uuid) { public List<MappingDataset> getByModelMappingDataset(UUID uuid) {
return queryFactory return queryFactory
@@ -116,4 +176,200 @@ public class ModelDetailRepositoryImpl implements ModelDetailRepositoryCustom {
.where(modelMasterEntity.uuid.eq(uuid)) .where(modelMasterEntity.uuid.eq(uuid))
.fetchOne(); .fetchOne();
} }
@Override
public List<ModelTrainMetrics> getModelTrainMetricResult(UUID uuid) {
ModelMasterEntity modelMasterEntity = findByModelByUUID(uuid);
if (modelMasterEntity == null) {
return List.of();
}
return queryFactory
.select(
Projections.constructor(
ModelTrainMetrics.class,
modelMetricsTrainEntity.epoch,
modelMetricsTrainEntity.iteration,
modelMetricsTrainEntity.loss,
modelMetricsTrainEntity.lr,
modelMetricsTrainEntity.durationTime))
.from(modelMetricsTrainEntity)
.where(modelMetricsTrainEntity.model.id.eq(modelMasterEntity.getId()))
.fetch();
}
@Override
public List<ModelValidationMetrics> getModelValidationMetricResult(UUID uuid) {
ModelMasterEntity modelMasterEntity = findByModelByUUID(uuid);
if (modelMasterEntity == null) {
return List.of();
}
return queryFactory
.select(
Projections.constructor(
ModelValidationMetrics.class,
modelMetricsValidationEntity.epoch,
modelMetricsValidationEntity.aAcc,
modelMetricsValidationEntity.mFscore,
modelMetricsValidationEntity.mPrecision,
modelMetricsValidationEntity.mRecall,
modelMetricsValidationEntity.mIou,
modelMetricsValidationEntity.mAcc,
modelMetricsValidationEntity.changedFscore,
modelMetricsValidationEntity.changedPrecision,
modelMetricsValidationEntity.changedRecall,
modelMetricsValidationEntity.unchangedFscore,
modelMetricsValidationEntity.unchangedPrecision,
modelMetricsValidationEntity.unchangedRecall))
.from(modelMetricsValidationEntity)
.where(modelMetricsValidationEntity.model.id.eq(modelMasterEntity.getId()))
.fetch();
}
@Override
public List<ModelTestMetrics> getModelTestMetricResult(UUID uuid) {
ModelMasterEntity modelMasterEntity = findByModelByUUID(uuid);
if (modelMasterEntity == null) {
return List.of();
}
return queryFactory
.select(
Projections.constructor(
ModelTestMetrics.class,
modelMetricsTestEntity.model1,
modelMetricsTestEntity.tp,
modelMetricsTestEntity.fp,
modelMetricsTestEntity.fn,
modelMetricsTestEntity.precisions,
modelMetricsTestEntity.recall,
modelMetricsTestEntity.f1Score,
modelMetricsTestEntity.accuracy,
modelMetricsTestEntity.iou,
modelMetricsTestEntity.detectionCount,
modelMetricsTestEntity.gtCount))
.from(modelMetricsTestEntity)
.where(modelMetricsTestEntity.model.id.eq(modelMasterEntity.getId()))
.fetch();
}
@Override
public ModelBestEpoch getModelTrainBestEpoch(UUID uuid) {
ModelMasterEntity modelMasterEntity = findByModelByUUID(uuid);
if (modelMasterEntity == null) {
return null;
}
return queryFactory
.select(
Projections.constructor(
ModelBestEpoch.class,
modelMetricsTrainEntity.epoch,
modelMetricsTrainEntity.loss,
modelMetricsValidationEntity.mFscore,
modelMetricsValidationEntity.mPrecision,
modelMetricsValidationEntity.mRecall,
modelMetricsValidationEntity.mIou,
modelMetricsValidationEntity.mAcc))
.from(modelMetricsTrainEntity)
.leftJoin(modelMetricsValidationEntity)
.on(
modelMetricsTrainEntity.model.eq(modelMetricsValidationEntity.model),
modelMetricsTrainEntity.epoch.eq(modelMetricsValidationEntity.epoch))
.where(
modelMetricsTrainEntity.model.id.eq(modelMasterEntity.getId()),
modelMetricsTrainEntity.epoch.eq(modelMasterEntity.getBestEpoch()))
.fetchOne();
}
@Override
public ModelFileInfo getModelTrainFileInfo(UUID uuid) {
return queryFactory
.select(
Projections.constructor(
ModelFileInfo.class,
modelMasterEntity
.packingState
.eq(TrainStatusType.COMPLETED.getId())
.coalesce(false),
modelMasterEntity.modelVer))
.from(modelMasterEntity)
.where(modelMasterEntity.uuid.eq(uuid))
.fetchOne();
}
@Override
public List<ModelProgressStepDto> findModelTrainProgressInfo(UUID uuid) {
ModelMasterEntity entity = findByModelByUUID(uuid);
if (entity == null) {
return List.of();
}
List<ModelProgressStepDto> steps = new ArrayList<>();
// 0단계 : 대기 상태
steps.add(
ModelProgressStepDto.builder()
.step(0)
.status(TrainStatusType.READY.getId())
.startTime(entity.getCreatedDttm())
.endTime(null)
.isError(false)
.build());
// 1단계 : Train/Validation 실행
boolean step1Active =
entity.getStep1StrtDttm() != null
&& !TrainStatusType.READY.getId().equals(entity.getStep1State());
if (step1Active) {
steps.add(
ModelProgressStepDto.builder()
.step(1)
.status(entity.getStep1State())
.startTime(entity.getStep1StrtDttm())
.endTime(entity.getStep1EndDttm())
.isError(TrainStatusType.ERROR.getId().equals(entity.getStep1State()))
.build());
}
// 2단계 : Test 실행
boolean step2Done = entity.getStep2State() != null;
if (step2Done) {
steps.add(
ModelProgressStepDto.builder()
.step(2)
.status(entity.getStep2State())
.startTime(entity.getStep2StrtDttm())
.endTime(entity.getStep2EndDttm())
.isError(TrainStatusType.ERROR.getId().equals(entity.getStep2State()))
.build());
}
// 3단계 : 패키징
boolean step3Done = entity.getPackingState() != null;
if (step3Done) {
steps.add(
ModelProgressStepDto.builder()
.step(3)
.status(entity.getPackingState())
.startTime(entity.getPackingStrtDttm())
.endTime(entity.getPackingEndDttm())
.isError(TrainStatusType.ERROR.getId().equals(entity.getPackingState()))
.build());
}
return steps;
}
@Override
public ModelMasterEntity findByModelBeforeId(Long beforeModelId) {
return queryFactory
.selectFrom(modelMasterEntity)
.where(modelMasterEntity.id.eq(beforeModelId))
.fetchOne();
}
} }

View File

@@ -1,7 +1,9 @@
package com.kamco.cd.training.postgres.repository.model; package com.kamco.cd.training.postgres.repository.model;
import com.kamco.cd.training.model.dto.ModelTrainMngDto; import com.kamco.cd.training.model.dto.ModelTrainMngDto;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.ListDto;
import com.kamco.cd.training.postgres.entity.ModelMasterEntity; import com.kamco.cd.training.postgres.entity.ModelMasterEntity;
import com.kamco.cd.training.train.dto.TrainRunRequest;
import java.util.Optional; import java.util.Optional;
import java.util.UUID; import java.util.UUID;
import org.springframework.data.domain.Page; import org.springframework.data.domain.Page;
@@ -14,9 +16,13 @@ public interface ModelMngRepositoryCustom {
* @param searchReq * @param searchReq
* @return * @return
*/ */
Page<ModelMasterEntity> findByModels(ModelTrainMngDto.SearchReq searchReq); Page<ListDto> findByModels(ModelTrainMngDto.SearchReq searchReq);
Optional<ModelMasterEntity> findByUuid(UUID uuid); Optional<ModelMasterEntity> findByUuid(UUID uuid);
Optional<ModelMasterEntity> findFirstByStatusCdAndDelYn(String statusCd, Boolean delYn); Optional<ModelMasterEntity> findFirstByStatusCdAndDelYn(String statusCd, Boolean delYn);
TrainRunRequest findTrainRunRequest(Long modelId);
Long findModelStep1InProgressCnt();
} }

View File

@@ -1,10 +1,20 @@
package com.kamco.cd.training.postgres.repository.model; package com.kamco.cd.training.postgres.repository.model;
import static com.kamco.cd.training.postgres.entity.QMemberEntity.memberEntity;
import static com.kamco.cd.training.postgres.entity.QModelConfigEntity.modelConfigEntity;
import static com.kamco.cd.training.postgres.entity.QModelHyperParamEntity.modelHyperParamEntity;
import static com.kamco.cd.training.postgres.entity.QModelMasterEntity.modelMasterEntity; import static com.kamco.cd.training.postgres.entity.QModelMasterEntity.modelMasterEntity;
import com.kamco.cd.training.common.enums.TrainStatusType;
import com.kamco.cd.training.model.dto.ModelTrainMngDto; import com.kamco.cd.training.model.dto.ModelTrainMngDto;
import com.kamco.cd.training.model.dto.ModelTrainMngDto.ListDto;
import com.kamco.cd.training.postgres.entity.ModelMasterEntity; import com.kamco.cd.training.postgres.entity.ModelMasterEntity;
import com.kamco.cd.training.postgres.entity.QModelMasterEntity;
import com.kamco.cd.training.train.dto.TrainRunRequest;
import com.querydsl.core.BooleanBuilder; import com.querydsl.core.BooleanBuilder;
import com.querydsl.core.types.Expression;
import com.querydsl.core.types.Projections;
import com.querydsl.core.types.dsl.Expressions;
import com.querydsl.jpa.impl.JPAQueryFactory; import com.querydsl.jpa.impl.JPAQueryFactory;
import java.util.List; import java.util.List;
import java.util.Optional; import java.util.Optional;
@@ -28,21 +38,62 @@ public class ModelMngRepositoryImpl implements ModelMngRepositoryCustom {
* @return * @return
*/ */
@Override @Override
public Page<ModelMasterEntity> findByModels(ModelTrainMngDto.SearchReq req) { public Page<ListDto> findByModels(ModelTrainMngDto.SearchReq req) {
QModelMasterEntity beforeModel = new QModelMasterEntity("beforeModel"); // alias
Expression<UUID> beforeModelUuid =
com.querydsl.jpa.JPAExpressions.select(beforeModel.uuid)
.from(beforeModel)
.where(beforeModel.id.eq(modelMasterEntity.beforeModelId));
Pageable pageable = req.toPageable(); Pageable pageable = req.toPageable();
BooleanBuilder builder = new BooleanBuilder(); BooleanBuilder builder = new BooleanBuilder();
if (req.getStatus() != null && !req.getStatus().isEmpty()) { if (req.getStatus() != null && !req.getStatus().isEmpty()) {
builder.and(modelMasterEntity.statusCd.eq(req.getStatus())); builder.and(
modelMasterEntity
.step1State
.eq(req.getStatus())
.or(modelMasterEntity.step2State.eq(req.getStatus())));
} }
if (req.getModelNo() != null && !req.getModelNo().isEmpty()) { if (req.getModelNo() != null && !req.getModelNo().isEmpty()) {
builder.and(modelMasterEntity.modelNo.eq(req.getModelNo())); builder.and(modelMasterEntity.modelNo.eq(req.getModelNo()));
} }
List<ModelMasterEntity> content = builder.and(modelMasterEntity.delYn.isFalse());
List<ListDto> content =
queryFactory queryFactory
.selectFrom(modelMasterEntity) .select(
Projections.constructor(
ListDto.class,
modelMasterEntity.id,
modelMasterEntity.uuid,
modelMasterEntity.modelVer,
modelMasterEntity.strtDttm,
modelMasterEntity.step1StrtDttm,
modelMasterEntity.step1EndDttm,
modelMasterEntity.step2StrtDttm,
modelMasterEntity.step2EndDttm,
modelMasterEntity.step1State,
modelMasterEntity.step2State,
modelMasterEntity.statusCd,
modelMasterEntity.trainType,
modelMasterEntity.modelNo,
modelMasterEntity.currentAttemptId,
modelMasterEntity.requestPath,
modelMasterEntity.packingState,
modelMasterEntity.packingStrtDttm,
modelMasterEntity.packingEndDttm,
modelConfigEntity.memo,
memberEntity.name,
beforeModelUuid))
.from(modelMasterEntity)
.innerJoin(modelConfigEntity)
.on(modelMasterEntity.id.eq(modelConfigEntity.model.id))
.leftJoin(memberEntity)
.on(modelMasterEntity.createdUid.eq(memberEntity.id))
.where(builder) .where(builder)
.offset(pageable.getOffset()) .offset(pageable.getOffset())
.limit(pageable.getPageSize()) .limit(pageable.getPageSize())
@@ -54,6 +105,10 @@ public class ModelMngRepositoryImpl implements ModelMngRepositoryCustom {
queryFactory queryFactory
.select(modelMasterEntity.count()) .select(modelMasterEntity.count())
.from(modelMasterEntity) .from(modelMasterEntity)
.innerJoin(modelConfigEntity)
.on(modelMasterEntity.id.eq(modelConfigEntity.model.id))
.leftJoin(memberEntity)
.on(modelMasterEntity.createdUid.eq(memberEntity.id))
.where(builder) .where(builder)
.fetchOne(); .fetchOne();
@@ -82,4 +137,75 @@ public class ModelMngRepositoryImpl implements ModelMngRepositoryCustom {
public Optional<ModelMasterEntity> findFirstByStatusCdAndDelYn(String statusCd, Boolean delYn) { public Optional<ModelMasterEntity> findFirstByStatusCdAndDelYn(String statusCd, Boolean delYn) {
return Optional.empty(); return Optional.empty();
} }
@Override
public TrainRunRequest findTrainRunRequest(Long modelId) {
return queryFactory
.select(
Projections.constructor(
TrainRunRequest.class,
modelMasterEntity.requestPath, // datasetFolder
modelMasterEntity.uuid, // outputFolder
modelHyperParamEntity.inputSize,
modelHyperParamEntity.cropSize,
modelHyperParamEntity.batchSize,
modelHyperParamEntity.gpuIds,
modelHyperParamEntity.gpuCnt,
modelHyperParamEntity.learningRate,
modelHyperParamEntity.backbone,
modelConfigEntity.epochCount,
modelHyperParamEntity.trainNumWorkers,
modelHyperParamEntity.valNumWorkers,
modelHyperParamEntity.testNumWorkers,
modelHyperParamEntity.trainShuffle,
modelHyperParamEntity.trainPersistent,
modelHyperParamEntity.valPersistent,
modelHyperParamEntity.dropPathRate,
modelHyperParamEntity.frozenStages,
modelHyperParamEntity.neckPolicy,
modelHyperParamEntity.classWeight,
modelHyperParamEntity.decoderChannels,
modelHyperParamEntity.weightDecay,
modelHyperParamEntity.layerDecayRate,
modelHyperParamEntity.ignoreIndex,
modelHyperParamEntity.ddpFindUnusedParams,
modelHyperParamEntity.numLayers,
modelHyperParamEntity.metrics,
modelHyperParamEntity.saveBest,
modelHyperParamEntity.saveBestRule,
modelHyperParamEntity.valInterval,
modelHyperParamEntity.logInterval,
modelHyperParamEntity.visInterval,
modelHyperParamEntity.rotProb,
modelHyperParamEntity.rotDegree,
modelHyperParamEntity.flipProb,
modelHyperParamEntity.exchangeProb,
modelHyperParamEntity.brightnessDelta,
modelHyperParamEntity.contrastRange,
modelHyperParamEntity.saturationRange,
modelHyperParamEntity.hueDelta,
Expressions.nullExpression(Integer.class),
Expressions.nullExpression(String.class),
modelHyperParamEntity.uuid))
.from(modelMasterEntity)
.leftJoin(modelHyperParamEntity)
.on(modelHyperParamEntity.id.eq(modelMasterEntity.hyperParamId))
.leftJoin(modelConfigEntity)
.on(modelConfigEntity.model.id.eq(modelMasterEntity.id))
.where(modelMasterEntity.id.eq(modelId))
.fetchOne();
}
@Override
public Long findModelStep1InProgressCnt() {
return queryFactory
.select(modelMasterEntity.id.count())
.from(modelMasterEntity)
.where(
modelMasterEntity
.step1State
.eq(TrainStatusType.IN_PROGRESS.getId())
.or(modelMasterEntity.step2State.eq(TrainStatusType.IN_PROGRESS.getId())))
.fetchOne();
}
} }

View File

@@ -0,0 +1,9 @@
package com.kamco.cd.training.postgres.repository.train;
import com.kamco.cd.training.postgres.entity.ModelMetricsTestEntity;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;
@Repository
public interface ModelTestMetricsJobRepository
extends JpaRepository<ModelMetricsTestEntity, Long>, ModelTestMetricsJobRepositoryCustom {}

View File

@@ -0,0 +1,24 @@
package com.kamco.cd.training.postgres.repository.train;
import com.kamco.cd.training.train.dto.ModelTrainMetricsDto.ModelMetricJsonDto;
import com.kamco.cd.training.train.dto.ModelTrainMetricsDto.ModelTestFileName;
import com.kamco.cd.training.train.dto.ModelTrainMetricsDto.ResponsePathDto;
import java.time.ZonedDateTime;
import java.util.List;
public interface ModelTestMetricsJobRepositoryCustom {
void updateModelMetricsTrainSaveYn(Long modelId, String stepNo);
List<ResponsePathDto> getTestMetricSaveNotYetModelIds();
void insertModelMetricsTest(List<Object[]> batchArgs);
ModelMetricJsonDto getTestMetricPackingInfo(Long modelId);
ModelTestFileName findModelTestFileNames(Long modelId);
void updatePackingStart(Long modelId, ZonedDateTime now);
void updatePackingEnd(Long modelId, ZonedDateTime now, String failSuccState);
}

View File

@@ -0,0 +1,171 @@
package com.kamco.cd.training.postgres.repository.train;
import static com.kamco.cd.training.postgres.entity.QModelMasterEntity.modelMasterEntity;
import static com.kamco.cd.training.postgres.entity.QModelMetricsTestEntity.modelMetricsTestEntity;
import static com.kamco.cd.training.postgres.entity.QModelMetricsTrainEntity.modelMetricsTrainEntity;
import com.kamco.cd.training.common.enums.TrainStatusType;
import com.kamco.cd.training.postgres.entity.ModelMetricsTestEntity;
import com.kamco.cd.training.train.dto.ModelTrainMetricsDto.ModelMetricJsonDto;
import com.kamco.cd.training.train.dto.ModelTrainMetricsDto.ModelTestFileName;
import com.kamco.cd.training.train.dto.ModelTrainMetricsDto.Properties;
import com.kamco.cd.training.train.dto.ModelTrainMetricsDto.ResponsePathDto;
import com.querydsl.core.types.Projections;
import com.querydsl.jpa.impl.JPAQueryFactory;
import java.time.ZonedDateTime;
import java.util.List;
import org.springframework.data.jpa.repository.support.QuerydslRepositorySupport;
import org.springframework.jdbc.core.JdbcTemplate;
public class ModelTestMetricsJobRepositoryImpl extends QuerydslRepositorySupport
implements ModelTestMetricsJobRepositoryCustom {
private final JPAQueryFactory queryFactory;
private final JdbcTemplate jdbcTemplate;
public ModelTestMetricsJobRepositoryImpl(
JPAQueryFactory queryFactory, JdbcTemplate jdbcTemplate) {
super(ModelMetricsTestEntity.class);
this.queryFactory = queryFactory;
this.jdbcTemplate = jdbcTemplate;
}
@Override
public void updateModelMetricsTrainSaveYn(Long modelId, String stepNo) {
queryFactory
.update(modelMasterEntity)
.set(
stepNo.equals("step1")
? modelMasterEntity.step1MetricSaveYn
: modelMasterEntity.step2MetricSaveYn,
true)
.where(modelMasterEntity.id.eq(modelId))
.execute();
}
@Override
public List<ResponsePathDto> getTestMetricSaveNotYetModelIds() {
return queryFactory
.select(
Projections.constructor(
ResponsePathDto.class,
modelMasterEntity.id,
modelMasterEntity.responsePath,
modelMasterEntity.uuid))
.from(modelMasterEntity)
.where(
modelMasterEntity.step2EndDttm.isNotNull(),
modelMasterEntity.step2State.eq(TrainStatusType.COMPLETED.getId()),
modelMasterEntity
.step2MetricSaveYn
.isNull()
.or(modelMasterEntity.step2MetricSaveYn.isFalse()))
.fetch();
}
@Override
public void insertModelMetricsTest(List<Object[]> batchArgs) {
// AS-IS
// String sql =
// """
// insert into tb_model_metrics_test
// (model_id, model, tp, fp, fn, precisions, recall, f1_score, accuracy, iou,
// detection_count, gt_count
// )
// values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
// """;
//
// jdbcTemplate.batchUpdate(sql, batchArgs);
// TO-BE: modelId, model(best_fscore_10) 같은 데이터가 있으면 update, 없으면 insert
String updateSql =
"""
UPDATE tb_model_metrics_test
SET tp=?, fp=?, fn=?, precisions=?, recall=?, f1_score=?, accuracy=?, iou=?,
detection_count=?, gt_count=?
WHERE model_id=? AND model=?
""";
String insertSql =
"""
INSERT INTO tb_model_metrics_test
(model_id, model, tp, fp, fn, precisions, recall, f1_score, accuracy, iou,
detection_count, gt_count)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""";
// row 단위 처리 (batch 안에서 upsert)
for (Object[] row : batchArgs) {
// row 순서: (model_id, model, tp, fp, fn, precisions, recall, f1_score, accuracy, iou,
// detection_count, gt_count)
int updated =
jdbcTemplate.update(
updateSql, row[2], row[3], row[4], row[5], row[6], row[7], row[8], row[9], row[10],
row[11], row[0], row[1]);
if (updated == 0) {
jdbcTemplate.update(insertSql, row);
}
}
}
@Override
public ModelMetricJsonDto getTestMetricPackingInfo(Long modelId) {
return queryFactory
.select(
Projections.constructor(
ModelMetricJsonDto.class,
modelMasterEntity.modelNo,
modelMasterEntity.modelVer,
Projections.constructor(
Properties.class,
modelMetricsTestEntity.f1Score,
modelMetricsTestEntity.precisions,
modelMetricsTestEntity.recall,
modelMetricsTestEntity.iou,
modelMetricsTrainEntity.loss)))
.from(modelMetricsTestEntity)
.innerJoin(modelMasterEntity)
.on(modelMetricsTestEntity.model.id.eq(modelMasterEntity.id))
.innerJoin(modelMetricsTrainEntity)
.on(
modelMetricsTestEntity.model.eq(modelMetricsTrainEntity.model),
modelMasterEntity.bestEpoch.eq(modelMetricsTrainEntity.epoch))
.where(modelMetricsTestEntity.model.id.eq(modelId))
.orderBy(modelMetricsTestEntity.createdDttm.desc())
.fetchFirst();
}
@Override
public ModelTestFileName findModelTestFileNames(Long modelId) {
return queryFactory
.select(
Projections.constructor(
ModelTestFileName.class, modelMetricsTestEntity.model1, modelMasterEntity.modelVer))
.from(modelMetricsTestEntity)
.innerJoin(modelMasterEntity)
.on(modelMetricsTestEntity.model.id.eq(modelMasterEntity.id))
.where(modelMetricsTestEntity.model.id.eq(modelId))
.fetchOne();
}
@Override
public void updatePackingStart(Long modelId, ZonedDateTime now) {
queryFactory
.update(modelMasterEntity)
.set(modelMasterEntity.packingStrtDttm, ZonedDateTime.now())
.set(modelMasterEntity.packingState, TrainStatusType.READY.getId())
.where(modelMasterEntity.id.eq(modelId))
.execute();
}
@Override
public void updatePackingEnd(Long modelId, ZonedDateTime now, String failSuccState) {
queryFactory
.update(modelMasterEntity)
.set(modelMasterEntity.packingEndDttm, ZonedDateTime.now())
.set(modelMasterEntity.packingState, failSuccState)
.where(modelMasterEntity.id.eq(modelId))
.execute();
}
}

View File

@@ -0,0 +1,7 @@
package com.kamco.cd.training.postgres.repository.train;
import com.kamco.cd.training.postgres.entity.ModelTrainJobEntity;
import org.springframework.data.jpa.repository.JpaRepository;
public interface ModelTrainJobRepository
extends JpaRepository<ModelTrainJobEntity, Long>, ModelTrainJobRepositoryCustom {}

View File

@@ -0,0 +1,17 @@
package com.kamco.cd.training.postgres.repository.train;
import com.kamco.cd.training.postgres.entity.ModelTrainJobEntity;
import java.util.List;
import java.util.Optional;
public interface ModelTrainJobRepositoryCustom {
int findMaxAttemptNo(Long modelId);
Optional<ModelTrainJobEntity> findLatestByModelId(Long modelId);
Optional<ModelTrainJobEntity> findByContainerName(String containerName);
void insertModelTestTrainingRun(Long modelId, Long jobId, int epoch);
List<ModelTrainJobEntity> findRunningJobs();
}

View File

@@ -0,0 +1,99 @@
package com.kamco.cd.training.postgres.repository.train;
import static com.kamco.cd.training.postgres.entity.QModelTestTrainingRunEntity.modelTestTrainingRunEntity;
import static com.kamco.cd.training.postgres.entity.QModelTrainJobEntity.modelTrainJobEntity;
import com.kamco.cd.training.common.enums.JobStatusType;
import com.kamco.cd.training.common.enums.JobType;
import com.kamco.cd.training.postgres.entity.ModelTrainJobEntity;
import com.kamco.cd.training.postgres.entity.QModelTrainJobEntity;
import com.querydsl.jpa.impl.JPAQueryFactory;
import jakarta.persistence.EntityManager;
import java.util.List;
import java.util.Optional;
import org.springframework.stereotype.Repository;
@Repository
public class ModelTrainJobRepositoryImpl implements ModelTrainJobRepositoryCustom {
private final JPAQueryFactory queryFactory;
public ModelTrainJobRepositoryImpl(EntityManager em) {
this.queryFactory = new JPAQueryFactory(em);
}
/** modelId의 attempt_no 최대값. (없으면 0) */
@Override
public int findMaxAttemptNo(Long modelId) {
QModelTrainJobEntity j = modelTrainJobEntity;
Integer max =
queryFactory.select(j.attemptNo.max()).from(j).where(j.modelId.eq(modelId)).fetchOne();
return max != null ? max : 0;
}
/**
* modelId의 최신 job 1건 (보통 id desc / queuedDttm desc 등) - attemptNo 기준으로도 가능하지만, 여기선 id desc가 가장
* 단순.
*/
@Override
public Optional<ModelTrainJobEntity> findLatestByModelId(Long modelId) {
QModelTrainJobEntity j = modelTrainJobEntity;
ModelTrainJobEntity job =
queryFactory.selectFrom(j).where(j.modelId.eq(modelId)).orderBy(j.id.desc()).fetchFirst();
return Optional.ofNullable(job);
}
@Override
public Optional<ModelTrainJobEntity> findByContainerName(String containerName) {
QModelTrainJobEntity j = modelTrainJobEntity;
ModelTrainJobEntity job =
queryFactory
.selectFrom(j)
.where(j.containerName.eq(containerName))
.orderBy(j.id.desc())
.fetchFirst();
return Optional.ofNullable(job);
}
@Override
public void insertModelTestTrainingRun(Long modelId, Long jobId, int epoch) {
Integer maxAttemptNo =
queryFactory
.select(modelTestTrainingRunEntity.attemptNo.max().coalesce(0))
.from(modelTestTrainingRunEntity)
.where(modelTestTrainingRunEntity.modelId.eq(modelId))
.fetchOne();
int nextAttemptNo = (maxAttemptNo == null ? 1 : maxAttemptNo + 1);
queryFactory
.insert(modelTestTrainingRunEntity)
.columns(
modelTestTrainingRunEntity.modelId,
modelTestTrainingRunEntity.attemptNo,
modelTestTrainingRunEntity.jobId,
modelTestTrainingRunEntity.epoch)
.values(modelId, nextAttemptNo, jobId, epoch)
.execute();
}
@Override
public List<ModelTrainJobEntity> findRunningJobs() {
return queryFactory
.select(modelTrainJobEntity)
.from(modelTrainJobEntity)
.where(
modelTrainJobEntity
.statusCd
.eq(JobStatusType.RUNNING.getId())
.and(modelTrainJobEntity.jobType.eq(JobType.TRAIN.getId())))
.orderBy(modelTrainJobEntity.id.desc())
.fetch();
}
}

View File

@@ -0,0 +1,9 @@
package com.kamco.cd.training.postgres.repository.train;
import com.kamco.cd.training.postgres.entity.ModelMetricsTrainEntity;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;
@Repository
public interface ModelTrainMetricsJobRepository
extends JpaRepository<ModelMetricsTrainEntity, Long>, ModelTrainMetricsJobRepositoryCustom {}

View File

@@ -0,0 +1,17 @@
package com.kamco.cd.training.postgres.repository.train;
import com.kamco.cd.training.train.dto.ModelTrainMetricsDto.ResponsePathDto;
import java.util.List;
public interface ModelTrainMetricsJobRepositoryCustom {
List<ResponsePathDto> getTrainMetricSaveNotYetModelIds();
void insertModelMetricsTrain(List<Object[]> batchArgs);
void updateModelMetricsTrainSaveYn(Long modelId, String stepNo);
void insertModelMetricsValidation(List<Object[]> batchArgs);
void updateModelSelectedBestEpoch(Long modelId, Integer epoch);
}

View File

@@ -0,0 +1,94 @@
package com.kamco.cd.training.postgres.repository.train;
import static com.kamco.cd.training.postgres.entity.QModelMasterEntity.modelMasterEntity;
import com.kamco.cd.training.common.enums.TrainStatusType;
import com.kamco.cd.training.postgres.entity.ModelMetricsTrainEntity;
import com.kamco.cd.training.train.dto.ModelTrainMetricsDto.ResponsePathDto;
import com.querydsl.core.types.Projections;
import com.querydsl.jpa.impl.JPAQueryFactory;
import java.util.List;
import org.springframework.data.jpa.repository.support.QuerydslRepositorySupport;
import org.springframework.jdbc.core.JdbcTemplate;
public class ModelTrainMetricsJobRepositoryImpl extends QuerydslRepositorySupport
implements ModelTrainMetricsJobRepositoryCustom {
private final JPAQueryFactory queryFactory;
private final JdbcTemplate jdbcTemplate;
public ModelTrainMetricsJobRepositoryImpl(
JPAQueryFactory queryFactory, JdbcTemplate jdbcTemplate) {
super(ModelMetricsTrainEntity.class);
this.queryFactory = queryFactory;
this.jdbcTemplate = jdbcTemplate;
}
@Override
public List<ResponsePathDto> getTrainMetricSaveNotYetModelIds() {
return queryFactory
.select(
Projections.constructor(
ResponsePathDto.class,
modelMasterEntity.id,
modelMasterEntity.responsePath,
modelMasterEntity.uuid))
.from(modelMasterEntity)
.where(
modelMasterEntity.step1EndDttm.isNotNull(),
modelMasterEntity.step1State.eq(TrainStatusType.COMPLETED.getId()),
modelMasterEntity
.step1MetricSaveYn
.isNull()
.or(modelMasterEntity.step1MetricSaveYn.isFalse()))
.fetch();
}
@Override
public void insertModelMetricsTrain(List<Object[]> batchArgs) {
String sql =
"""
insert into tb_model_metrics_train
(model_id, epoch, iteration, loss, lr, duration_time)
values (?, ?, ?, ?, ?, ?)
""";
jdbcTemplate.batchUpdate(sql, batchArgs);
}
@Override
public void updateModelMetricsTrainSaveYn(Long modelId, String stepNo) {
queryFactory
.update(modelMasterEntity)
.set(
stepNo.equals("step1")
? modelMasterEntity.step1MetricSaveYn
: modelMasterEntity.step2MetricSaveYn,
true)
.where(modelMasterEntity.id.eq(modelId))
.execute();
}
@Override
public void insertModelMetricsValidation(List<Object[]> batchArgs) {
String sql =
"""
insert into tb_model_metrics_validation
(model_id, epoch, a_acc, m_fscore, m_precision, m_recall, m_iou, m_acc, changed_fscore, changed_precision, changed_recall,
unchanged_fscore, unchanged_precision, unchanged_recall
)
values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""";
jdbcTemplate.batchUpdate(sql, batchArgs);
}
@Override
public void updateModelSelectedBestEpoch(Long modelId, Integer epoch) {
queryFactory
.update(modelMasterEntity)
.set(modelMasterEntity.bestEpoch, epoch)
.where(modelMasterEntity.id.eq(modelId))
.execute();
}
}

View File

@@ -0,0 +1,216 @@
package com.kamco.cd.training.train;
import com.kamco.cd.training.config.api.ApiResponseDto;
import com.kamco.cd.training.train.service.DataSetCountersService;
import com.kamco.cd.training.train.service.TestJobService;
import com.kamco.cd.training.train.service.TrainJobService;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.Parameter;
import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.responses.ApiResponses;
import io.swagger.v3.oas.annotations.tags.Tag;
import java.util.UUID;
import lombok.RequiredArgsConstructor;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@Tag(name = "학습 실행 API", description = "모델학습관리 > 학습 실행 API")
@RequiredArgsConstructor
@RestController
@RequestMapping("/api/train")
public class TrainApiController {
private final TrainJobService trainJobService;
private final TestJobService testJobService;
private final DataSetCountersService dataSetCountersService;
@Operation(summary = "학습 실행", description = "학습 실행 API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "실행 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = String.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@PostMapping("/run/{uuid}")
public ApiResponseDto<String> run(
@Parameter(description = "uuid", example = "80a0e544-36ed-4999-b705-97427f23337d")
@PathVariable
UUID uuid) {
Long modelId = trainJobService.getModelIdByUuid(uuid);
trainJobService.enqueue(modelId);
return ApiResponseDto.ok("ok");
}
@Operation(summary = "학습 재실행", description = "학습 재실행 API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "재실행 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = String.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@PostMapping("/restart/{uuid}")
public ApiResponseDto<String> restart(
@Parameter(description = "uuid", example = "80a0e544-36ed-4999-b705-97427f23337d")
@PathVariable
UUID uuid) {
Long modelId = trainJobService.getModelIdByUuid(uuid);
Long jobId = trainJobService.restart(modelId);
return ApiResponseDto.ok("ok");
}
@Operation(summary = "학습 이어하기", description = "학습 이어하기 API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "이어하기 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = String.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@PostMapping("/resume/{uuid}")
public ApiResponseDto<String> resume(
@Parameter(description = "uuid", example = "80a0e544-36ed-4999-b705-97427f23337d")
@PathVariable
UUID uuid) {
Long modelId = trainJobService.getModelIdByUuid(uuid);
Long jobId = trainJobService.resume(modelId);
return ApiResponseDto.ok("ok");
}
@Operation(summary = "학습 취소", description = "학습 취소 API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "취소 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = String.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@PostMapping("/cancel/{uuid}")
public ApiResponseDto<String> cancel(
@Parameter(description = "uuid", example = "80a0e544-36ed-4999-b705-97427f23337d")
@PathVariable
UUID uuid) {
Long modelId = trainJobService.getModelIdByUuid(uuid);
trainJobService.cancel(modelId);
return ApiResponseDto.ok("ok");
}
@Operation(summary = "test 실행", description = "test 실행 API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "test 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = String.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@PostMapping("/test/run/{epoch}/{uuid}")
public ApiResponseDto<String> run(
@Parameter(description = "best 에폭", example = "1") @PathVariable int epoch,
@Parameter(description = "uuid", example = "80a0e544-36ed-4999-b705-97427f23337d")
@PathVariable
UUID uuid) {
Long modelId = trainJobService.getModelIdByUuid(uuid);
testJobService.enqueue(modelId, uuid, epoch);
return ApiResponseDto.ok("ok");
}
@Operation(summary = "test 학습 취소", description = "학습 취소 API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "취소 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = String.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@PostMapping("/test/cancel/{uuid}")
public ApiResponseDto<String> cancelTest(
@Parameter(description = "uuid", example = "80a0e544-36ed-4999-b705-97427f23337d")
@PathVariable
UUID uuid) {
Long modelId = trainJobService.getModelIdByUuid(uuid);
testJobService.cancel(modelId);
return ApiResponseDto.ok("ok");
}
@Operation(summary = "데이터셋 tmp 파일생성", description = "데이터셋 tmp 파일생성 API")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "데이터셋 tmp 파일생성 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = String.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@PostMapping("/create-tmp/{uuid}")
public ApiResponseDto<UUID> createTmpFile(
@Parameter(description = "model uuid", example = "80a0e544-36ed-4999-b705-97427f23337d")
@PathVariable
UUID uuid) {
return ApiResponseDto.ok(trainJobService.createTmpFile(uuid));
}
@Operation(summary = "getCount", description = "getCount 서버 로그확인")
@ApiResponses(
value = {
@ApiResponse(
responseCode = "200",
description = "데이터셋 tmp 파일생성 성공",
content =
@Content(
mediaType = "application/json",
schema = @Schema(implementation = String.class))),
@ApiResponse(responseCode = "400", description = "잘못된 검색 조건", content = @Content),
@ApiResponse(responseCode = "500", description = "서버 오류", content = @Content)
})
@GetMapping(path = "/counts/{uuid}", produces = MediaType.APPLICATION_JSON_VALUE)
public ApiResponseDto<String> getCount(
@Parameter(description = "uuid", example = "e22181eb-2ac4-4100-9941-d06efce25c49")
@PathVariable
UUID uuid) {
Long modelId = trainJobService.getModelIdByUuid(uuid);
return ApiResponseDto.ok(dataSetCountersService.getCount(modelId));
}
}

View File

@@ -0,0 +1,22 @@
package com.kamco.cd.training.train.dto;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
@Getter
@Setter
@AllArgsConstructor
@NoArgsConstructor
public class EvalRunRequest {
private String uuid;
private int epoch; // best_changed_fscore_epoch_1.pth
private Integer timeoutSeconds;
private String datasetFolder;
private String outputFolder;
public String getOutputFolder() {
return this.outputFolder.toString();
}
}

View File

@@ -0,0 +1,25 @@
package com.kamco.cd.training.train.dto;
import java.time.ZonedDateTime;
import java.util.Map;
import lombok.AllArgsConstructor;
import lombok.Getter;
@Getter
@AllArgsConstructor
public class ModelTrainJobDto {
private Long id;
private Long modelId;
private Integer attemptNo;
private String statusCd;
private Integer exitCode;
private String errorMessage;
private String containerName;
private Map<String, Object> paramsJson;
private ZonedDateTime queuedDttm;
private ZonedDateTime startedDttm;
private ZonedDateTime finishedDttm;
private Integer totalEpoch;
private Integer currentEpoch;
}

View File

@@ -0,0 +1,15 @@
package com.kamco.cd.training.train.dto;
/** 학습 실행이 예약되었음을 알리는 이벤트 객체 */
public class ModelTrainJobQueuedEvent {
private final Long jobId;
public ModelTrainJobQueuedEvent(Long jobId) {
this.jobId = jobId;
}
public Long getJobId() {
return jobId;
}
}

View File

@@ -0,0 +1,24 @@
package com.kamco.cd.training.train.dto;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public class ModelTrainLinkDto {
private Long modelId;
private String trainType;
private String modelNo;
private Long datasetId;
private String targetClassCd;
private String comparePath;
private String targetPath;
private String labelPath;
private String geoJsonPath;
private String datasetUid;
}

View File

@@ -0,0 +1,59 @@
package com.kamco.cd.training.train.dto;
import com.fasterxml.jackson.annotation.JsonProperty;
import io.swagger.v3.oas.annotations.media.Schema;
import java.util.Properties;
import java.util.UUID;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
public class ModelTrainMetricsDto {
@Schema(name = "ResponsePathDto", description = "AI 결과 저장된 path 경로 정보")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public static class ResponsePathDto {
private Long modelId;
private String responsePath;
private UUID uuid;
}
@Getter
@AllArgsConstructor
public static class ModelMetricJsonDto {
@JsonProperty("cd_model_type")
private String cdModelType;
@JsonProperty("model_version")
private String modelVersion;
private Properties properties;
}
@Getter
@AllArgsConstructor
public static class Properties {
@JsonProperty("f1_score")
private Float f1Score;
private Float precision;
private Float recall;
private Float loss;
private Double iou;
}
@Getter
@AllArgsConstructor
public static class ModelTestFileName {
private String bestEpochFileName;
private String modelVersion;
}
}

View File

@@ -0,0 +1,94 @@
package com.kamco.cd.training.train.dto;
import java.util.UUID;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public class TrainRunRequest {
// ========================
// 기본
// ========================
private String datasetFolder;
private UUID outputFolder;
private String inputSize;
private String cropSize;
private Integer batchSize;
private String gpuIds;
private Integer gpus;
private Double learningRate;
private String backbone;
private Integer epochs;
// ========================
// Data
// ========================
private Integer trainNumWorkers;
private Integer valNumWorkers;
private Integer testNumWorkers;
private Boolean trainShuffle;
private Boolean trainPersistent;
private Boolean valPersistent;
// ========================
// Model Architecture
// ========================
private Double dropPathRate;
private Integer frozenStages;
private String neckPolicy;
private String classWeight;
private String decoderChannels;
// ========================
// Loss & Optimization
// ========================
private Double weightDecay;
private Double layerDecayRate;
private Integer ignoreIndex;
private Boolean ddpFindUnusedParams;
private Integer numLayers;
// ========================
// Evaluation
// ========================
private String metrics;
private String saveBest;
private String saveBestRule;
private Integer valInterval;
private Integer logInterval;
private Integer visInterval;
// ========================
// Augmentation
// ========================
private Double rotProb;
private String rotDegree;
private Double flipProb;
private Double exchangeProb;
private Integer brightnessDelta;
private String contrastRange;
private String saturationRange;
private Integer hueDelta;
// ========================
// 실행 타임아웃
// ========================
private Integer timeoutSeconds;
private String resumeFrom;
private UUID uuid;
public String getOutputFolder() {
return String.valueOf(this.outputFolder);
}
public String getUuid() {
return String.valueOf(this.uuid);
}
}

Some files were not shown because too many files have changed in this diff Show More