Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
T
transaction-log-desensitization
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
廖鸣韬
transaction-log-desensitization
Commits
8ac48834
Commit
8ac48834
authored
Nov 21, 2020
by
谢森
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
<dev> 判断时间段内无数据
parent
0cf4e4e5
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
25 additions
and
20 deletions
+25
-20
src/main/java/com/zorkdata/desensitization/hadoop/HdfsLogDesensitization.java
...rkdata/desensitization/hadoop/HdfsLogDesensitization.java
+25
-20
No files found.
src/main/java/com/zorkdata/desensitization/hadoop/HdfsLogDesensitization.java
View file @
8ac48834
...
...
@@ -95,7 +95,7 @@ public class HdfsLogDesensitization implements Serializable {
this
.
hdfsUri
=
String
.
valueOf
(
conf
.
get
(
ConfigConstants
.
HDFS_URI
)).
trim
();
this
.
hdfsUser
=
String
.
valueOf
(
conf
.
get
(
ConfigConstants
.
HDFS_USER
)).
trim
();
this
.
hdfsSrc
=
hdfsUri
+
String
.
valueOf
(
conf
.
get
(
ConfigConstants
.
HDFS_SRC
)).
trim
();
this
.
hdfsDest
=
hdfsUri
+
String
.
valueOf
(
conf
.
get
(
ConfigConstants
.
HDFS_DEST
)).
trim
();
this
.
hdfsDest
=
hdfsUri
+
String
.
valueOf
(
conf
.
get
(
ConfigConstants
.
HDFS_DEST
)).
trim
();
this
.
core
=
String
.
valueOf
(
conf
.
get
(
ConfigConstants
.
CORE
)).
trim
();
this
.
startTime
=
String
.
valueOf
(
conf
.
get
(
ConfigConstants
.
START_TIME
));
this
.
endTime
=
String
.
valueOf
(
conf
.
get
(
ConfigConstants
.
END_TIME
));
...
...
@@ -276,28 +276,33 @@ public class HdfsLogDesensitization implements Serializable {
FileSystem
fileSystem
=
null
;
List
<
String
>
betweenDate
=
DateUtil
.
getBetweenDate
(
startTime
,
endTime
);
List
<
String
>
dateList
=
DateUtil
.
date2date
(
betweenDate
);
try
{
fileSystem
=
FileSystem
.
get
(
new
URI
(
hdfsUri
),
conf
,
hdfsUser
);
for
(
String
item
:
dateList
)
{
path
=
hdfsSrc
+
item
;
List
<
String
>
hdfsLogFiles
=
getHdfsLogFilesByPath
(
fileSystem
,
path
);
logFiles
.
addAll
(
hdfsLogFiles
);
}
}
catch
(
IOException
e
)
{
log
.
error
(
String
.
valueOf
(
e
));
}
catch
(
InterruptedException
e
)
{
log
.
error
(
String
.
valueOf
(
e
));
}
catch
(
URISyntaxException
e
)
{
log
.
error
(
String
.
valueOf
(
e
));
}
finally
{
if
(
null
!=
fileSystem
)
{
try
{
fileSystem
.
close
();
}
catch
(
IOException
e
)
{
log
.
error
(
String
.
valueOf
(
e
));
if
(
dateList
.
size
()
>
0
)
{
try
{
fileSystem
=
FileSystem
.
get
(
new
URI
(
hdfsUri
),
conf
,
hdfsUser
);
for
(
String
item
:
dateList
)
{
path
=
hdfsSrc
+
item
;
List
<
String
>
hdfsLogFiles
=
getHdfsLogFilesByPath
(
fileSystem
,
path
);
logFiles
.
addAll
(
hdfsLogFiles
);
}
}
catch
(
IOException
e
)
{
log
.
error
(
String
.
valueOf
(
e
));
}
catch
(
InterruptedException
e
)
{
log
.
error
(
String
.
valueOf
(
e
));
}
catch
(
URISyntaxException
e
)
{
log
.
error
(
String
.
valueOf
(
e
));
}
finally
{
if
(
null
!=
fileSystem
)
{
try
{
fileSystem
.
close
();
}
catch
(
IOException
e
)
{
log
.
error
(
String
.
valueOf
(
e
));
}
}
}
}
else
{
log
.
warn
(
"{} -- {} 时间段内无数据,请注意时间范围"
,
startTime
,
endTime
);
}
return
logFiles
;
}
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment