why not just do both?
why not just do both? our product stores all the logs raw in flats files on the file system, we don't use databases for keeping the logs in, this allows you to scale massively (ingestion limit is that of the correlation engine and disk bandwidth). You then just need an efficient search crawler and use of metadata so search performance is good too. Issue is if you every need to pull the logs for court and you have messed with them (i.e. normalized them and stuffed them into a DB) then your chain of custody is broken. Best of both worlds means parsed out normalisation so I don't have to remember that Juniper calls source ip srcIP and Cisco SourceIP, but the original logs under the covers for grepping if you need.
Posted by cones688 on May 7, 2015, 12:21 PM
Tags
Related issues
- This is my biggest problem with agents, they constantly defensively code for cases that have no chance of happening and in most cases a p...
- I wish there was something like LEAN for biology.
- The issue is really the extreme difficulty of programming them, especially for anything sequential (rather than combinatorial).
- I probably could have beared with it if it was only limited to programming, but it showing up in creative fields has made me feel all sor...
- I hate that I'm in deep enough to know what you mean.
- The biggest problem with “AI” is that big corps are shoving it down our throats.
Raise real-world problems, find people facing the same issue, and discuss solutions.6,500 issues0 comments
reliability (Other)
This is my biggest problem with agents, they constantly defensively code for cases that have no chance of happening and in most cases a p...
I wish there was something like LEAN for biology.
The issue is really the extreme difficulty of programming them, especially for anything sequential (rather than combinatorial).
I probably could have beared with it if it was only limited to programming, but it showing up in creative fields has made me feel all sor...
I hate that I'm in deep enough to know what you mean.
The biggest problem with “AI” is that big corps are shoving it down our throats.
in the middel of a coding session with 5.6 sol.
[Hero Run] 535B-A23B on 18T tokens
why not just do both?
why not just do both? our product stores all the logs raw in flats files on the file system, we don't use databases for keeping the logs in, this allows you to scale massively (ingestion limit is that of the correlation engine and disk bandwidth). You then just need an efficient search crawler and use of metadata so search performance is good too. Issue is if you every need to pull the logs for court and you have messed with them (i.e. normalized them and stuffed them into a DB) then your chain of custody is broken. Best of both worlds means parsed out normalisation so I don't have to remember that Juniper calls source ip srcIP and Cisco SourceIP, but the original logs under the covers for grepping if you need.
Comments (0)
No comments yet. Start the discussion.